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Why  GAO  Did  This  Study 

VA  has  26  ongoing  medical-facility 
construction  projects  intended,  for 
example,  to  provide  improved  care  to 
veterans  returning  from  Afghanistan 
and  Iraq.  GAO  has  previously  reported 
on  VA’s  weaknesses  in  managing 
major  projects.  Congress  continues  to 
have  questions  about  VA’s  project 
management  practices  and  mandated 
that  VA  outsource  to  other  federal 
entities  the  design  and  construction  of 
certain  ongoing  projects  and  future 
projects  costing  $100  million  or  more. 

In  response  to  a  2016  defense 
authorization,  this  report  assesses  1) 
VA’s  actions  since  2013  to  address 
challenges  managing  projects  costing 
$100  million  or  more  and  2) 
opportunities  for  improvements  in 
managing  these  projects,  particularly 
VA’s  medical  facility  in  Denver,  CO,  the 
only  project  outsourced  to  USACE  that 
is  under  construction.  GAO  reviewed 
reports  on  VA  management  of  projects, 
interviewed  VA  and  USACE  officials, 
and  visited  five  projects  to  compare 
their  management  to  VA  policies  and 
procedures.  Selected  projects  include 
the  most  costly,  those  in  various 
stages  of  construction  and  projects 
managed  by  VA  and  USACE.  GAO 
also  analyzed  the  estimated  cost  and 
schedule  of  the  Denver  project  for 
adherence  to  best  practices. 

What  GAO  Recommends 

VA  should  (1)  establish  a  mechanism 
to  monitor  change  orders,  (2)  develop 
a  reliable  activation  cost  estimate  for 
the  Denver  project,  and  (3)  clarify 
policies  on  integrating  schedules.  VA 
concurred  with  our  recommendations. 
VA  and  USACE  provided  technical 
comments  which  we  incorporated  as 
appropriate. 

View  GAO-17-70.  For  more  information, 
contact  Dave  Wise  at  (202)  512-5731  or 
wised@gao.gov 


What  GAO  Found 

The  Department  of  Veterans  Affairs  (VA)  has  taken  steps  to  address  challenges 
in  managing  projects  to  build  medical  facilities.  In  response  to  statutory 
requirements  and  additional  congressional  direction,  VA  is  outsourcing 
management  of  certain  such  projects  to  the  U.S.  Army  Corps  of  Engineers 
(USACE).  As  of  October  2016,  VA  had  23  ongoing  projects  costing  $100  million 
or  more.  VA  and  USACE  have  entered  into  interagency  agreements  for  12  of 
these  23  projects.  The  agreements  entail  USACE's  managing  the  projects  while 
VA  retains  responsibility  for  their  overall  completion,  including  activation  (making 
the  facility  ready  for  full  operation  after  construction,  such  as  adding  medical 
equipment).  VA  has  also  revised  policies  and  procedures  for  managing  projects 
not  outsourced  to  USACE,  such  as  streamlining  the  change  order  process,  or 
approving  changes  to  a  facility’s  design. 

VA  still  has  opportunities  to  improve  tracking  change  orders  for  major  projects 
and  estimating  cost  and  schedules  for  the  $1 .675-billion  Denver  construction 
project — the  only  USACE-managed  project  under  construction  thus  far: 

•  Specifically,  while  VA  has  issued  guidelines  to  streamline  the  change  order 
process  and  plans  to  collect  data  on  time  frames,  it  lacks  a  mechanism  to 
systematically  collect  or  monitor  data  on  time  frames  to  process  change 
orders.  Although  VA's  contract  management  software  collects  information  on 
dates  change  orders  were  initiated  and  approved,  VA  does  not  use  the 
software  to  determine  if  changes  are  approved  within  the  required  time. 
Further,  it  does  not  currently  track  reasons  for  change  orders,  such  as 
whether  VA  medical  staff  requested  them.  VA  plans  to  replace  this  software 
with  a  system  that  records  this  information.  Although  procurement  of  this 
system  has  been  delayed,  VA  intends  to  implement  a  system  to  monitor  time 
frames  by  March  2017.  However,  it  is  not  yet  clear  how  VA  plans  to  use  new 
information  it  collects  to  oversee  change  orders  because  VA  lacks  a 
mechanism  to  oversee  and  monitor  changes  to  a  facility’s  design  as  a  project 
progresses.  Without  such  a  mechanism,  VA  cannot  determine  how 
processing  timeframes  and  design  changes  affect  costs  and  schedules  and 
thus  is  at  risk  for  unexpected  cost  increases  and  schedule  delays. 

•  In  assessing  VA's  medical  facility  project  in  Denver,  GAO  found  opportunities 
to  improve  cost  estimates  and  schedules.  VA's  activation  cost  increased  from 
$272  million  in  2012  to  $341  million  currently.  However,  the  current  estimate 
is  not  reliable;  VA  officials  could  not  provide  information  on  how  they 
developed  it  and  GAO  could  not  determine  if  it  meets  criteria  in  the  GAO 
Cost  Estimating  and  Assessment  Guide.  Further,  GAO’s  analysis  showed 
that  the  construction  and  activation  schedules  are  not  integrated,  so  that  the 
construction  schedule’s  milestones  do  not  align  with  the  activation  schedule. 
Leading  practices  and  VA  policies  both  call  for  integrating  such  schedules  to 
help  ensure  projects’  successful  and  timely  completion.  However,  VA's 
policies  to  integrate  such  schedules  are  inconsistent  and  unclear.  The 
combined  problems  with  the  activation  cost  estimate  and  schedule 
integration  put  Denver’s  VA  medical  facility  at  risk  of  further  cost  increases 
and  delays.  Without  reliable  information  on  activation  costs  and  schedules  for 
the  Denver  project,  VA  has  no  assurance  that  the  schedules  are  realistic  and 
that  current  funding  will  suffice  to  complete  construction  and  activation. 


United  States  Government  Accountability  Office 


Contents 


Letter 

1 

Background 

VA  Has  Taken  Steps  Intended  to  Address  Administration  and 

4 

Oversight  Challenges  of  Medical-Facility  Construction  Projects 
Improvements  Managing  Change  Orders  and  Estimating  Project 

8 

Costs  and  Schedules  Could  Help  VA  Address  Continuing  Cost 
Increases  and  Schedule  Delays 

13 

Conclusions 

32 

Recommendations  for  Executive  Action 

33 

Agency  Comments 

33 

Appendix  1 

Scope  and  Methodology 

36 

Appendix  II 

Additional  Information  on  Medical  Center  Projects  Outsourced  to 

U.S.  Army  Corps  of  Engineers 

39 

Appendix  III 

Changes  in  Costs  and  Products  for  Major  Medical-Facility  Projects 

40 

Appendix  IV 

Comparison  of  the  Denver  Construction’s  Cost  Estimate  with  Best 
Practices  for  Cost  Estimating 

43 

Appendix  V 

Comparison  of  the  Denver  Construction’s  Schedule  Estimate  with 
Best  Practices 

47 

Appendix  VI 

Comments  from  the  Department  of  Veterans  Affairs 

49 

Appendix  VII 

GAO  Contact  and  Staff  Acknowledgments 

60 

Related  GAO  products 

61 

Page  i 


GAO-17-70  VA  Construction 


Tables 


Table  1 :  Other  Department  of  Veterans  Affairs’  (VA)  Actions  to 

Improve  Management  of  Medical  Facility  Projects  13 

Table  2:  Changes  in  Costs  and  Completion  Time  Frames  between 
November  2012  and  October  2016  for  Selected 
Department  of  Veterans  Affairs’  (VA)  Medical-Facility 
Construction  Projects  15 

Table  3:  GAO’s  Assessment  of  the  Cost  Estimate  to  Complete 
Construction  of  the  Department  of  Veterans  Affairs 
Denver  Medical  Center  Compared  to  the  Four 
Characteristics  of  High-Ouality,  Reliable  Cost  Estimates  20 
Table  4:  GAO  Assessment  of  Schedule  Estimate  to  Complete 

Construction  of  the  Department  of  Veterans  Affairs’  (VA) 

Denver  Medical  Center  Compared  to  the  Four 
Characteristics  of  High-Ouality,  Reliable  Schedule 


Estimates  26 

Table  5:  List  of  Department  of  Veterans  Affairs  Medical  Center 

Projects  Outsourced  to  the  U.S.  Army  Corps  of  Engineers 
(USACE)  for  Construction  39 

Table  6:  Changes  in  Costs  and  Completion  Time  Frames  for 
Department  of  Veterans  Affairs’  (VA)  Major  Medical- 
Facility  Projects  between  November  2012  and  October 
2016  40 

Table  7:  GAO  Assessment  of  the  Cost  Estimate  to  Complete 

Construction  of  the  Department  of  Veterans  Affairs  (VA) 

Denver  Medical  Center  Compared  to  the  Four 
Characteristics  of  High-Ouality,  Reliable  Cost  Estimates  44 

Table  8:  GAO’s  Assessment  of  the  Schedule  Estimate  to 

Complete  Construction  of  the  Department  of  Veterans 
Affairs’  (VA)  Denver  Medical  Center,  Compared  to  the 
Four  Characteristics  of  High-Ouality,  Reliable  Cost 
Estimates  47 


Figures 

Figure  1 :  Phases  of  a  Department  of  Veterans  Affairs’  (VA)  Project 

for  Medical  Facility  Construction.  6 

Figure  2:  Denver  Department  of  Veterans  Affairs  Medical  Center 

Project  (November  2016)  9 


Page  ii 


GAO-17-70  VA  Construction 


Abbreviations 


ACBM 

CFM 

eCMS 

IMS 

OMB 

PMP 

USACE 

VA 

VHA 


Activation  Cost  Budget  Model 

Office  of  Construction  and  Facilities  Management 

Electronic  Contract  Management  System 

integrated  master  schedule 

Office  of  Management  and  Budget 

project  management  plans 

U.S.  Army  Corps  of  Engineers 

Department  of  Veterans  Affairs 

Veterans  Health  Administration 


This  is  a  work  of  the  U.S.  government  and  is  not  subject  to  copyright  protection  in  the 
United  States.  The  published  product  may  be  reproduced  and  distributed  in  its  entirety 
without  further  permission  from  GAO.  However,  because  this  work  may  contain 
copyrighted  images  or  other  material,  permission  from  the  copyright  holder  may  be 
necessary  if  you  wish  to  reproduce  this  material  separately. 


Page  iii 


GAO-17-70  VA  Construction 
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Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


March  7,  2017 
Congressional  Addressees 

The  Department  of  Veterans  Affairs  (VA)  operates  one  of  the  largest 
health  care  systems  in  the  country  with  over  1,700  sites  serving  almost  9- 
million  veterans  in  2015.  VA  has  been  building  new  medical  facilities  to 
replace  facilities  that  were  built  decades  ago  and  to  provide  veterans 
returning  from  Afghanistan  and  Iraq  with  specialized  care.  VA  is  required 
to  submit  a  prospectus  to  the  House  and  Senate  Committees  on 
Veterans’  Affairs  when  it  proposes  to  build,  renovate,  or  acquire  any 
medical  facilities  estimated  to  cost  more  than  $10  million,1  known  as 
“major  medical  facility  projects.”  We  have  previously  reported  on 
significant  cost  overruns  on  VA’s  projects  to  build  major  medical  facilities 
as  well  as  weaknesses  in  managing  these  projects.2  Specifically,  in  2013, 
we  reported  on  cost  overruns  totaling  $1.5  billion  on  four  major  medical- 
facility  construction  projects  in  Denver,  CO;  Las  Vegas,  NV;  New  Orleans, 
LA;  and  Orlando,  FL.3  Congress  has  raised  questions  about  VA’s 
administration  and  oversight  of  these  projects  and  passed  several  laws 
related  to  VA’s  construction  of  major  medical  facilities:  (1)  in  2015, 
Congress  passed  and  the  president  signed  the  Department  of  Veterans 
Affairs  Expiring  Authorities  Act  of  2015,  (2)  the  National  Defense 
Authorization  Act  for  Fiscal  Year  2016,  and  (3)  the  Consolidated 
Appropriations  Act,  201 6. 4  Collectively,  these  three  laws  require  VA  to 
contract  with  other  federal  entities  to  provide  full  project-management 
services  for  the  design  and  construction  of  certain  ongoing  construction 
projects  with  a  total  estimated  cost  of  $100  million  or  more.5  Furthermore, 
the  three  laws  collectively  require  that  VA  contract  out  management  of 


138  U.S.C.  §  8104. 

2GAO,  VA  Construction:  Additional  Actions  Needed  to  Decrease  Delays  and  Lower  Costs 
of  Major  Medical-Facility  Projects,  GAO-13-302  (Washington.  D.C.:  April  2013). 

3GAO-1 3-302. 

4Pub.  L.  No.  114-58,  §  502,  129  Stat.  530,  537-38;  Pub.  L.  No.  114-92,  129  Stat.  726, 

1020  (2015);  Pub.  L.  No.  114-113,  129  Stat.  2242,  2691-92  (2015),  respectively. 

5ln  addition,  the  explanatory  statement  accompanying  the  Consolidated  Appropriations 
Act,  2016  specified  seven  ongoing  projects  for  which  VA  was  directed  to  outsource  design 
and  construction  management.  These  seven  projects  are  in  Alameda,  CA;  American  Lake, 
WA;  Livermore,  CA;  Long  Beach,  CA;  Louisville,  KY;  San  Francisco,  CA;  and  West  Los 
Angeles,  CA. 


Page  1 


GAO-17-70  VA  Construction 


any  new  projects  costing  $100  million  or  more.6  All  of  these  projects  that 
are  contracted  out  are  known  as  “super  construction  projects.”  VA  has 
chosen  the  U.S.  Army  Corps  of  Engineers  (USACE)  to  manage  these 
projects.  VA  will  continue  to  manage  other  ongoing  construction  projects 
costing  $100  million  or  more  not  specified  by  the  three  2015  laws. 

Regardless  of  whether  a  VA  medical  facility  project  is  managed  by 
USACE  or  VA,  all  such  projects  are  funded  through  VA  appropriations. 
Additionally,  VA  will  still  be  responsible  for  “activating”  all  construction 
projects — a  process  by  which  VA  identifies,  plans,  and  manages  the 
steps  to  bring  a  newly  built  medical  facility  into  operation,  such  as 
purchasing  medical  equipment  and  hiring  new  staff. 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2016  included  a 
provision  requiring  us  to  report  on  VA’s  oversight  and  administration  of 
the  design  and  construction  of  major  medical-facility  projects.  Separately 
in  2015,  House  Veterans  Affairs  Committee  requesters  asked  us  to 
review  the  same  issues.  In  November  2016  we  provided  an  interim 
briefing  to  Senate  and  House  Veterans  Affairs  Committees  as  well  as  the 
Senate  and  House  Appropriations  Subcommittees  on  Military 
Construction,  Veterans  Affairs,  and  Related  Agencies  to  meet  our 
mandated  reporting  date.  This  report  assesses  VA’s  administration  and 
oversight  of  the  medical  facility  construction  projects  outsourced  to 
USACE  and  those  that  VA  still  manages.7  Specifically,  we  reviewed 

•  actions  VA  has  taken  since  2013  to  address  challenges  in  its 
administration  and  oversight  of  the  highest  cost  medical-facility 
construction  projects  (those  costing  $100  million  or  more),  and 

•  what  opportunities,  if  any,  exist  for  VA  to  make  additional 
improvements  to  its  administration  and  oversight  of  these  projects, 


6More  specifically,  the  construction  management  requirements  in  Department  of  Veterans 
Affairs  Expiring  Authorities  Act  of  2015  explicitly  apply  to  the  VA  medical  facility 
construction  project  in  Denver,  CO,  as  well  as  any  super  construction  project  authorized 
on  or  after  September  30, 201 5.  Pub.  L.  No.  114-58,  §  502,  129  Stat.  530,  537-38.  The 
construction  management  requirements  in  the  National  Defense  Authorization  Act  for 
Fiscal  Year  2016  apply  to  VA  medical  facility  projects  of  more  than  $100  million  authorized 
after  November  25,  2015.  Pub.  L.  No.  114-92,  129  Stat.  726,  1020  (2015).  In  addition,  the 
Consolidated  Appropriation  Act,  2016  provided  that  $649  million  of  VA’s  Fiscal  Year  2016 
appropriation  would  remain  unavailable  until  VA  enters  into  an  agreement  with  an 
appropriate  non-VA  federal  entity  for  projects  with  a  total  estimated  cost  of  $100  million  or 
more.  Pub.  L.  No.  114-113,  129  Stat.  2242,  2691-92  (2015). 

7AII  of  the  projects  that  we  focus  on  in  this  report  cost  $100  million  or  more. 
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particularly  at  the  VA  medical  facility  in  Denver,  CO,  the  only  USACE- 
managed  project  that  is  currently  under  construction. 

To  determine  the  actions  VA  has  taken  since  2013  to  address  challenges 
in  the  administration  and  oversight  of  medical-facility  construction  projects 
costing  $100  million  or  more,  we  reviewed  the  following:  (1)  prior  GAO, 

VA  Office  of  Inspector  General  and  USACE  reports  on  VA’s  management 
of  these  projects  and  (2)  VA’s  policies  and  guidance  on  project 
administration  and  oversight,  particularly  those  policies  and  guidances 
implemented  or  revised  since  we  last  reviewed  VA’s  construction  of 
medical  facilities.8  We  also  interviewed  VA  headquarters  and  USACE 
officials  and  selected  5  of  26  construction  projects  for  in-depth  case  study 
reviews  in  Denver,  CO;  Louisville,  KY;  New  Orleans,  LA;  Palo  Alto,  CA; 
and  St.  Louis,  MO.9  These  projects  were  among  the  most  costly  and  at  a 
variety  of  stages  in  the  construction  process  (see  fig.  1  on  pg.  5).  In  our 
sample,  we  also  included  projects  VA  was  managing  as  well  as  those 
USACE  was  managing.  For  each  project,  we  reviewed  construction 
documents  and  examined  cost  and  schedule  data.  We  also  interviewed 
VA  and,  where  applicable,  USACE  staff  responsible  for  managing  design 
and  construction  and  design  and  construction  contractors,  and  VA 
medical  center  staff,  and  liaison  with  representatives  of  local  veterans 
services  organizations.  The  information  from  our  selected  projects  is 
illustrative  and  cannot  be  generalized  to  sites  agency-wide.  To  determine 
how  VA  could  improve  its  administration  and  oversight  of  medical  facility 
construction  projects  costing  $100  million  or  more,  we  also  compared 
management  of  the  projects  we  reviewed  to  VA’s  policies  and  procedures 
to  determine  the  extent  to  which  they  were  followed.  In  addition,  we 
analyzed  cost  and  schedule  data  for  the  Denver,  CO,  project;  this  is  the 
only  project  that  USACE  is  managing  that  is  currently  under 
construction.10  We  discussed  Denver’s  activation  cost  and  schedule  with 


8GAO-1 3-302. 

9Of  these  26  projects,  23  are  estimated  to  cost  $100  million  or  more.  The  remaining  3  are 
estimated  to  cost  less  than  $100  million.  The  selected  projects  are  new  medical  facilities 
being  built  in  Denver,  CO;  Louisville,  KY;  and  New  Orleans,  LA;  ambulatory  care  and 
polytrauma-blind  rehabilitation  facilities  being  built  in  Palo  Alto,  CA;  and  a  facility 
improvement  project  in  St.  Louis,  MO.  The  Denver  project  is  actually  located  in  Aurora, 

CO,  but  because  it  is  replacing  the  facility  in  Denver  and  is  frequently  referred  to  as  the 
Denver  project;  we  will  refer  to  it  as  the  Denver  project  for  purposes  of  this  report.  VA  is 
managing  the  New  Orleans,  Palo  Alto,  and  St.  Louis  projects,  while  USACE  is  managing 
the  Denver  project  and  has  agreed  to  manage  the  Louisville  project. 

i  n 

The  Denver  project  is  the  largest  of  all  of  VA’s  projects  in  terms  of  cost  and  thus  the  only 
one  for  which  we  assessed  cost  and  schedule  estimates. 
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VA’s  local  and  headquarters  staff,  as  these  two  activation  factors  were 
not  integrated  within  the  construction  estimate  and  schedule.  We 
assessed  the  reliability  of  the  Denver  construction  estimate  and  schedule 
data  through  interviews  with  knowledgeable  VA  staff  and  a  review  for 
completeness  and  any  unexpected  values  and  determined  that  the  data 
were  sufficiently  reliable  for  the  purpose  of  our  reporting  objectives.  We 
compared  VA’s  and  USACE’s  process  for  estimating  the  cost  and 
schedule  of  the  Denver  project  with  GAO  best  practices.  Specifically,  we 
compared  the  process  used  to  estimate  the  Denver  project’s  construction 
cost  estimate  to  best  practices  identified  in  the  GAO  Cost  Estimating  and 
Assessment  Guided  Similarly,  we  compared  the  process  used  to 
develop  the  Denver  project’s  schedule  estimate  with  GAO’s  Schedule 
Assessment  Guide,  which  defines  best  practices  related  to  four 
characteristics — comprehensive,  well-constructed,  credible,  and 
controlled — of  high-quality,  reliable  schedule  estimates).12  Appendix  1 
contains  a  more  detailed  description  of  our  scope  and  methodology. 

We  conducted  this  performance  audit  from  January  201 6  to  March  201 7 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 

Background 

VA  has  pressing  infrastructure  needs  and  has  struggled  to  make  progress 
in  addressing  them.  Many  of  VA’s  facilities  were  built  decades  ago  and 
were  designed  for  an  inpatient-driven  health  care  system  and  do  not  align 
with  VA’s  current  wellness  approach  to  provide  health  care  through  an 
integrated  system  emphasizing  a  full  continuum  of  care,  in  particular 

11  GAO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs  (Supersedes  GAO-07-1 134SPJ,  GAO-09-3SP 
(Washington,  D.C.:  Mar.  2,  2009).  This  guide  identifies  best  practices  across  the  federal 
government  that  are  the  basis  for  a  high-quality,  reliable  cost  estimate. 

12GAO,  Schedule  Assessment  Guide:  Best  Practices  for  Project  Schedules,  GAO-16-89G 
(Washington,  D.C.:  Dec.  22,  2015).  The  GAO’s  Schedule  Assessment  Guide  presents  the 
scheduling  concepts  introduced  in  the  Cost  Estimating  and  Assessment  Guide  as  10 
leading  practices  associated  with  developing  and  maintaining  a  reliable,  high-quality 
schedule.  The  leading  practices  were  developed  in  conjunction  with  government  and 
industry  experts  in  the  schedule-estimating  community.  GAO's  Schedule  Assessment 
Guide  serves  also  to  present  guiding  principles  for  our  auditors  in  evaluating  the  economy, 
efficiency,  and  effectiveness  of  government  programs. 
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outpatient  care.  Additionally,  as  we  reported  in  201 3, 13  new  or  expanded 
facilities  are  needed  to  accommodate  veterans  returning  from  Iraq  and 
Afghanistan  who  need  specialized  treatment.  To  meet  the  health  care 
needs  of  veterans,  VA  has  23  construction  projects  under  way,  costing 
$100  million  or  more.  As  noted  previously,  VA  is  required  to  submit  a 
prospectus  to  the  House  and  Senate  Committees  on  Veterans’  Affairs 
that  contains  information  about  each  planned  medical-facility  project 
expected  to  cost  $10  million  or  more.14  This  information  includes  an  initial 
estimate  of  the  overall  cost  and,  in  some  cases,  a  completion  date  for  the 
project. 

VA  classifies  the  phases  of  construction  projects  it  manages  in  the 
following  terms:  (1)  initial  planning;  (2)  design  of  the  facility;  (3) 
construction;  and  (4)  activation.  (See  fig.  1  for  more  details.)  VA’s  Office 
of  Construction  and  Facilities  Management  (CFM)  in  conjunction  with 
VA’s  Veterans  Health  Administration  (VHA)  is  responsible  for  planning, 
designing,  and  constructing  new  medical  facilities,  including  medical 
facility  projects  costing  more  than  $100  million  that  are  not  managed  by 
USACE.15  VHA  is  responsible  for  activating  new  medical  facilities.  As 
figure  1  indicates,  activation  activities,  such  as  purchasing  equipment  and 
hiring  staff,  begin  during  design,  continue  through  construction,  and  end 
when  the  facility  is  fully  operational.  VA’s  Activations  Office  supports 
activation  efforts  by  issuing  guidance  and  offering  some  services, 
including  cost-estimating  services. 


13GAO-1 3-302. 

1438  U.S.C.  §  8104. 

15CFM  executes  these  projects  under  the  oversight  of  the  Office  of  Acquisition,  Logistics, 
and  Construction. 
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Figure  1 :  Phases  of  a  Department  of  Veterans  Affairs’  (VA)  Project  for  Medical  Facility  Construction. 


Planning 


VA  analyzes  its  needs  for  a  major 
medical  facility  and  conducts  studies 
to  define  the  facility’s  scope. 


•  VA  brings  the  facility  into  full  planned  operations  by  conducting 
activities  such  as  purchasing  and  installing  furniture  and 
medical  equipment  and  hiring  new  staff  for  the  facility. 


3/ 

v;  Design 


•  VA  hires  an  architectural  and  engineering 
firm  to  design  the  facility  and  develop  a 
cost  estimate  to  complete  the  project. 


•  VA  awards  a  construction  contract  to  a  contractor 
that  then  develops  a  detailed  construction  schedule 
and  constructs  the  facility. 


VA  reviews  the  schedule  and  assigns  engineers  to 
work  onsite  and  oversee  the  facility’s  construction 
until  it  is  complete. 


Source:  GAO  analysis  of  VA  information.  |  GAO-17-70 

VA  staff  at  various  organizational  levels  are  responsible  for  managing  the 
construction  of  medical-facility  projects.  Contracting  officers,  occasionally 
located  on-site  to  support  the  senior  resident  engineer  in  executing  the 
construction  contract,  are  ultimately  responsible  for  managing  the 
execution  of  construction  contracts;  other  onsite  staff — such  as  the 
project  executive  and  resident  engineers — oversee  the  actual 
construction.  In  some  instances,  staff  from  CFM’s  regional  offices  and 
VA’s  headquarters,  including  attorneys  in  the  Office  of  General  Counsel, 
provide  assistance  to  the  project  team.  According  to  VA  officials,  the  VA’s 
medical  center  staff,  such  as  the  medical  center  director,  are  involved  in 
facility  design  and  are  responsible  for  monitoring  construction  to  ensure 
that  the  facility  will  meet  veterans’  needs. 


According  to  USACE  officials,  USACE  acts  as  the  design  and 
construction  agent  on  projects  it  manages  and  its  construction  process 
includes  four  phases  that  are  similar  to  VA’s  process: 

•  Planning  and  Programming:  The  agency  that  USACE  supports — in 
this  case,  VA — identifies  the  need  for  the  project  and  its  scope. 
USACE  provides  technical  support  and  advice  as  needed. 

•  Design:  USACE  develops  criteria  for  the  project  and  engineering  and 
architectural  details  as  well  as  major  contract  procurement  and  other 
legal  documents  and  schedules.  USACE  takes  the  lead  in  this  phase; 
VA  is  directly  integrated  into  the  project  team. 
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•  Construction:  USACE  awards  and  administers  the  construction 
contract  and  is  involved  in  activities  such  as  project  management, 
engineering,  contracting,  and  legal  counsel.  USACE  takes  the  lead  in 
this  phase;  VA  is  directly  involved  in  the  project  team. 

•  Operation  and  Maintenance:  VA  is  responsible  for  operations  and 
maintenance  once  construction  is  complete,  although  USACE  may 
provide  technical  assistance  as  needed. 

As  we  previously  noted  in  our  2013  report,  most  construction  projects 
require  some  degree  of  change  to  the  facility  design  as  the  project 
progresses.  Typically,  organizations  have  a  process  to  initiate  and 
implement  these  changes  through  “change  orders.”16  Changes  can  occur 
for  a  variety  of  reasons,  including  design  errors,  unforeseen  site 
conditions,  and  changes  in  medical  practice  and  safety  requirements,  as 
well  as  changes  to  improve  equipment.  In  general,  government  contracts 
contain  a  changes  clause  that  permits  the  contracting  officer  to  make 
changes,  in  designated  areas,  within  the  general  scope  of  the  contract.17 
Contractors  can  also  request  changes  to  the  contract.  Agency  guidance 
specifies  milestones  for  change  orders,  depending  on  their  dollar  value.18 

In  our  2013  report,  we  found  significant  cost  increases  and  delays  for  the 
four  projects  we  reviewed.19  We  attributed  those  cost  increases  and 
delays  to  various  factors,  including  changes  to  veterans’  health  care 
needs,  site  acquisition  issues,  and  problems  in  VA’s  management  of 
medical  facility  projects.  We  made  recommendations  intended  to  improve 
VA’s  management  of  its  major  construction  projects  (specifically, 
problems  with  managing  change  orders  and  defining  VA  officials’  roles 
and  responsibilities).  These  recommendations  and  VA’s  responses  are 


16GAO-1 3-302.  These  changes  are  also  called  contract  modifications.  For  purposes  of  this 
report  we  will  refer  to  them  as  “change  orders.” 

1748  C.F.R§  43.201. 

1  ft 

VA,  Contract  Modification  Handbook  for  Resident  Engineers  in  Field  Offices:  Appendix 
III:  Modification  Milestones  (Version  1.0,  Aug.  29,  2013). 

19GAO-1 3-302. 
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discussed  later  in  this  report.  In  2015,  USACE  also  reviewed  the  same 
four  VA  medical  center  projects,  with  findings  similar  to  ours.20 


VA  Has  Taken  Steps 
Intended  to  Address 
Administration  and 
Oversight  Challenges 
of  Medical-Facility 
Construction  Projects 


Since  2013,  VA  Has  Taken 
Steps  toward  Jointly 
Managing  the  Projects  It 
Has  Outsourced  to 
USACE  in  Response  to  a 
Statutory  Mandate 


In  response  to  requirements  in  several  federal  statutes,  VA  is  in  the 
process  of  outsourcing  the  acquisition,  design  and  construction  of  certain 
medical  facility  projects  costing  $100  million  or  more  to  USACE. 
According  to  VA  staff,  VA  selected  USACE  because  of  its  engineering 
and  construction  experience.  Although  the  Denver  project  currently  is  the 
only  project  under  construction  that  jointly  involves  both  VA  and  USACE, 
the  agencies  have  established  an  interagency  agreement  that  provides 
general  terms  and  conditions  for  future  projects.21 


Further,  as  of  October,  2016,  VA  and  USACE  had  developed  interagency 
agreements  for  1 1  projects  in  addition  to  the  Denver  project  that  delegate 
contract  administration  authority  to  USACE.22  These  agreements  include 
the  value  of  the  services  USACE  will  provide  and  define  USACE’s  and 


20U.S.  Army  Corps  of  Engineers,  Major  Medical  Construction ,  United  States  Department 
of  Veterans  Affairs ,  A  Diagnostic  Assessment  by  the  United  States  Army  Corps  of 
Engineers ,  North  Las  Vegas  Medical  Center  ( July  15,  2015);  U.S.  Army  Corps  of 
Engineers,  Major  Medical  Construction ,  United  States  Department  of  Veterans  Affairs,  A 
Diagnostic  Assessment  by  the  United  States  Army  Corps  of  Engineers,  New  Orleans 
Medical  Center  ( July  15,  2015);  U.S.  Army  Corps  of  Engineers,  Major  Medical 
Construction,  United  States  Department  of  Veterans  Affairs,  A  Diagnostic  Assessment  by 
the  United  States  Army  Corps  of  Engineers,  Orlando  Medical  Center  (Aug.  17,  2015);  and 
U.S.  Army  Corps  of  Engineers,  Major  Medical  Construction,  United  States  Department  of 
Veterans  Affairs,  A  Diagnostic  Assessment  by  the  United  States  Army  Corps  of 
Engineers,  Denver-Aurora  Medical  Center  (June  8,  2015). 

21VA  and  USACE  plan  to  develop  more  specific  agreements  for  each  medical  facility 
project  USACE  agrees  to  manage. 

22For  a  list  of  VA  outsourced  and  managed  projects  and  status  see  appendix  II. 
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VA’s  respective  roles  and  responsibilities.  According  to  USACE  officials, 
for  each  medical  facility  construction  project  USACE  manages,  it  will 
provide  planning,  site  acquisition,  facility  design,  construction 
management,  and  contract  administration  services.  VA  will  maintain 
responsibility  for  informing  Congress  and  other  agencies  on  the  status  of 
projects’  cost  and  schedule,  although  USACE  may  provide  information 
regarding  the  project  upon  request.  In  October  2015,  USACE  began 
actively  managing  one  of  the  projects,  the  Denver  VA  Medical  Center 
project  (see  fig.  2),  which  was  about  78  percent  complete  as  of  October, 
2016,  according  to  VA.  As  part  of  managing  this  project,  USACE  awarded 
a  new  contract  to  the  construction  contractor  to  complete  the  remaining 
work  on  the  project.  The  other  projects  that  USACE  is  responsible  for 
managing  have  not  yet  reached  the  construction  phase.  According  to 
USACE  officials,  because  the  other  projects  USACE  is  taking  over  are  at 
varying  stages  of  completion,  the  extent  of  USACE’s  involvement  in  each 
project  may  vary.23 


Figure  2:  Denver  Department  of  Veterans  Affairs  Medical  Center  Project  (November  2016) 


Source:  U.S.  Army  Corps  of  Engineers.  |  GAO-17-70 


VA  and  USACE  have  also  taken  the  following  additional  steps  to 
coordinate  their  efforts  on  construction  projects  USACE  will  manage: 


no 

In  addition  to  taking  over  management  of  the  Denver,  CO,  project,  USACE  has  agreed 
to  take  over  management  of  the  Louisville,  KY,  project. 
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•  VA  and  USACE  developed  an  Enterprise  Program  Management  Plan 
to  provide  a  framework  and  consistent  approach  for  USACE  to 
support  VA’s  design  and  construction  program  in  the  future.  The  plan 
formalizes  USACE’s  role  and  aligns  with  the  underlying  goal  of  the 
agencies’  interagency  agreement  to  deliver  medical  facility  projects 
within  cost  and  on  schedule.  In  comments  on  a  draft  of  this  report,  VA 
reported  that  it  and  USACE  executed  the  plan  in  December  of  2016. 

•  VA  officials  stated  that,  in  the  future,  VA’s  CFM  workforce  will  be 
integrated  with  the  USACE  team  during  construction  to  (1)  enable 
workers  to  pass  along  institutional  knowledge  VA  staff  gained  while 
managing  the  Denver  project  and  (2)  collaborate  and  gain  familiarity 
with  USACE  methods.  For  example,  at  the  Denver  site,  eight  VA  CFM 
staff — most  of  whom  are  resident  engineers — are  collaborating  with 
USACE’s  team  to  learn  USACE’s  processes  and  share  knowledge  of 
VA’s  standards,  while  USACE  staff  are  responsible  for  overseeing  the 
contractors.  In  the  future,  VA  officials  expect  to  have  four  to  five  CFM 
staff  on-site  on  each  project  USACE  manages. 

•  VA  and  USACE  officials  told  us  they  established  “building  teams”  to 
help  the  two  agencies’  staff  work  together  toward  project  completion. 
Building  teams  are  comprised  of  VA,  USACE,  and  contractor  staff  and 
meet  regularly  to  chart  and  track  progress,  build  and  maintain  trust, 
and  resolve  issues  in  a  timely  manner.  For  example,  if  the  building 
team  discusses  and  reaches  agreement  on  a  change  order,  all  parties 
are  aware  of  the  change  at  the  same  time,  thus  expediting  resolution. 

•  VA  is  also  collaborating  with  USACE  staff  in  Denver  to  facilitate 
integrated  project  coordination  and  the  execution  of  contract 
administration,  according  to  VA  officials.  VA  officials  said  that 
integrating  CFM  staff  with  USACE  in  Denver  and  on  future  projects 
will  help  to  incorporate  USACE  practices  and  lessons  learned  on 
projects  that  VA  will  continue  to  manage.  USACE  also  provided 
Denver  CFM  senior  resident  engineers  and  resident  engineers  a  3- 
day  training  workshop  on  USACE  construction-quality  management 
processes.  According  to  VA,  these  participants  will  form  a  cadre  of 
CFM  staff  to  work  on  all  future  projects. 

•  VA  has  also  made  arrangements  to  reimburse  USACE  for  managing 
projects.  According  to  the  interagency  agreement  that  specifies 
general  terms  and  conditions,  VA  will  reimburse  USACE  for  costs  to 
manage  these  projects.  These  reimbursements  include  costs  related 
to  overseeing  the  design  and  construction  of  the  projects,  and  when 
applicable,  cost  for  activities  related  to  the  initial  assessment  and 
acceptance  of  VA’s  previous  work  in  design  or  construction,  or  both. 
VA  officials  told  us  that  each  project  will  continue  to  have  four  to  five 
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VA  staff  who  will  perform  such  tasks  as  interacting  with  medical  center 
staff  to  ensure  that  VA  design  requirements  are  met  and  to  coordinate 
changes  during  construction.  They  added  that,  therefore,  this  process 
will  entail  additional  personnel  and  layers  of  review  and  noted  that 
previously,  VA  had  four  staff  on-site  during  medical-facility 
construction  projects,  a  number  that  was  too  few  when  compared  with 
USACE’s  staffing  model.  The  VA  officials  further  noted  that  VA  does 
expect  to  increase  staff  on-site  at  projects  to  improve  safety  and 
control  project  cost  and  delivery  time. 


VA  Has  Taken  Some  Steps 
to  Address  Challenges 
Managing  Medical  Facility 
Projects 

•  Medical  Equipment  Planners:  Our  201 3  report  recommended  that  VA 
develop  and  implement  guidance  to  assign  medical  equipment 
planners  to  medical-construction  projects  costing  more  than  $10 
million.24  This  assignment  would  help  VA  better  respond  to  changes  in 
technology  and  equipment.  In  August  2013,  VA  issued  a  policy 
memorandum  providing  such  guidance.  The  memorandum  also 
requires  the  hiring  of  a  medical  equipment  specialist  through  the 
architectural  and  engineering  firm  responsible  for  designing  the 
project  when  construction  projects  need  medical  equipment  installed 
during  construction  when  VA  manages  construction.25  VA  officials  at 
our  selected  projects  indicated  that  VA  has  improved  its  capabilities 
for  medical  facilities’  planning,  including  equipment  planning,  to  better 
ensure  coordination  with  healthcare  staff.  For  example,  the  Palo  Alto 
medical  facility  has  a  permanent  staff  of  equipment  planners  and 
architects,  which  CFM  staff  said  will  reduce  the  number  of  design 
changes  and  change  orders  during  construction;  however,  as 
discussed  later  in  this  report,  VA  does  not  have  a  mechanism  in  place 
to  determine  if  these  steps  have  reduced  design  changes. 

•  Project  Roles  and  Responsibilities:  VA  also  responded  to  our 
recommendation  to  improve  VA  staff  communication  with  construction 
contractors  and  architectural  and  engineering  firms,  particularly  to 
clarify  roles  and  responsibilities  related  to  change  orders.  We  found  in 


VA  implemented  the  recommendations  we  made  in  our  2013  report 
regarding  the  management  of  the  four  largest  medical  facilities  projects  at 
the  time — all  costing  well  over  $100  million — and  other  identified 
challenges,  by  revising  the  following  policies  and  procedures: 


24GAO-1 3-302. 

25Department  of  Veterans  Affairs,  Office  of  Construction  &  Facilities  Management, 
Architectural  Design  Manual  (Aug.  1, 2014). 
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2013  that  a  lack  of  clear  communication  with  contractors  contributed 
to  schedule  delays  and  cost  increases  for  medical-facility  projects. 
Contractors  were  sometimes  confused  about  which  CFM  staff  were 
responsible  for  various  aspects  of  project  oversight.  In  September 
2013,  VA  implemented  procedures  to  communicate  to  contractors  the 
roles  and  responsibilities  of  VA  officials  who  manage  medical  facility 
construction  projects,  including  the  change  order  process.  These 
procedures  include  requirements  to  develop  a  communications  plan 
and  matrix  to  assure  clear  and  consistent  communications  with  all 
parties.  The  communications  plan  must  address,  among  other  issues, 
regular  project  communication,  such  as  meetings  and  in-progress 
reviews;  frequency  and  method  of  communication  (e.g.,  e-mail, 
phone);  and  stakeholder  roles  and  responsibilities.  On  the  three 
selected  projects  we  reviewed  that  VA  managed,  contractors  said 
they  had  established  good  working  agreements  with  CFM  to 
communicate  and  resolve  issues.  For  example,  VA  and  contractor 
officials  said  they  conduct  frequent  walk-throughs  and  weekly 
meetings  on  all  projects. 

•  Streamlining  change  order  approvals:  VA  took  several  steps  to 
respond  to  our  2013  recommendation  that  VA  streamline  its  change 
order  approval  process.26  First,  in  August  2013,  VA  issued  the 
Contract  Modification  Handbook  that  established  processing  time 
frames  for  change  orders  on  construction  contracts.27  For  example, 
the  Handbook  states  that  change  orders  under  $100,000  in  value 
should  be  processed  in  60  days.  Furthermore,  VA  raised  the  threshold 
for  change  orders  needing  legal  review  to  those  with  a  cost  of 
$700,000  or  more.  Previously,  all  change  orders  over  $100,000 
needed  legal  review,  resulting  in  delays  caused  by  the  length  of  the 
legal  review  process  and  the  large  number  of  changes  needing 
review.  Additionally,  in  March  2015,  VA  authorized  certain  regional 
officials  to  approve  change  orders  of  up  to  $2  million;  VA’s  Central 
Office  previously  handled  these  approvals.28  Although  VA  officials  told 
us  these  changes  have  helped  streamline  the  process,  as  discussed 


26GAO-1 3-302. 

27Department  of  Veterans  Affairs,  Contract  Modification  Handbook  for  Resident  Engineers 
in  Field  Offices  (Aug.  29,  2013).  In  commenting  on  a  draft  of  this  report,  VA  noted  that  it 
has  since  updated  this  handbook  and  expects  to  finalize  and  distribute  the  new  version  by 
the  end  of  the  second  quarter  of  fiscal  year  201 7. 

28ln  commenting  on  a  draft  of  this  report,  VA  stated  that  it  has  since  raised  these  limits. 
Currently,  the  contracting  officer’s  authority  to  approve  contract  actions,  including  change 
orders,  has  increased  to  $5  million. 


Page  12 


GAO-17-70  VA  Construction 


later,  VA  does  not  monitor  the  extent  that  change  orders  are 
processed  according  to  established  timelines.  As  a  result,  VA  does 
not  know  if  the  time  frames  for  processing  change  orders  have 
actually  improved. 

As  part  of  VA’s  overall  efforts  to  improve  management  of  medical  facility 
projects,  VA  also  made  other  changes  to  its  guidance  that  apply  to  all 
medical-facilities  projects  costing  $10  million  or  more  that  are  managed 
by  VA).  These  changes  are  summarized  in  table  1 . 


Table  1:  Other  Department  of  Veterans  Affairs’  (VA)  Actions  to  Improve  Management  of  Medical  Facility  Projects 

Action 

Description  of  action 

Issued  guidance  on  Framework  Principles  for  the 
Delivery  of  Major  Construction  Projects 

In  September  2013,  VA  published  its  guidance  to  assist  medical  staff  with 
activating  new  medical  facilities. 

Issued  guidance  on  Major  Construction  Projects — 
Roles  and  Responsibilities 

In  September  2015,  VA  issued  guidance  that  describes  the  roles  and 
responsibilities  of  medical  and  CFM  staff  to  enhance  the  delivery  of  projects. 

Issued  guidance  on  Foundation  of  Project  Design 
Principles 

In  November  201 5,  VA  published  principles  for  architectural  form  and  style 
specifying  that  designs  should  avoid  costly  and  unwarranted  architectural  and 
engineering  embellishments  and  unnecessary  construction  and  maintenance 
expenses. 

Issued  guidance  on  Changes  requested  by  VA  medical 
staff 

In  May  2016,  VA  clarified  CFM  responsibilities  and  authorities  for  evaluating 
and  approving  changes  requested  by  medical  centers  after  construction  has 
begun. 

Source:  GAO  Analysis  of  VA  information.  |  GAO-17-70 

Improvements 
Managing  Change 
Orders  and 
Estimating  Project 
Costs  and  Schedules 
Could  Help  VA 
Address  Continuing 
Cost  Increases  and 
Schedule  Delays 


Page  13 


GAO-17-70  VA  Construction 


Cost  Increases  and 
Schedule  Delays  Persist 
at  Major  Medical-Facility 
Projects 


Some  VA  major  medical-facility  projects  continue  to  experience  cost 
increases  and  schedule  delays  similar  to  those  that  we  found  in  our 
previous  review  of  VA’s  major  construction  projects.29  Of  the  26  ongoing 
construction  projects,30 10  projects  had  no  cost  increases  between 
November  2012  and  October  2016,  and  4  had  cost  decreases  ranging 
from  2.3  percent  to  16.4  percent.31  However,  10  projects  did  experience 
cost  increases.  For  example,  the  Denver,  CO,  project  has  experienced  a 
cost  increase  of  over  100  percent  since  November  2012,  and  the  San 
Francisco  project  has  experienced  a  cost  increase  of  about  54  percent. 
These  10  projects  have  a  combined  cost  increase  of  $1.3  billion.32 
Similarly,  13  projects  experienced  schedule  delays  between  November 
2012  and  October  201 6. 33  For  example,  the  New  Orleans  project  has 
been  delayed  by  almost  2  years.  Table  2  shows  changes  in  estimated 
costs  and  schedules  between  November  2012  and  October  2016  for  the 
five  projects  we  selected  for  this  review.  See  appendix  III  for  changes  in 
cost  and  schedule  for  all  26  ongoing  major  medical-facilities  projects. 
Below,  we  describe  issues  related  to  managing  change  orders  and 
estimating  costs  and  scheduling  that  could  be  contributing  to  the 
persistence  of  cost  increases  and  schedule  delays. 


29GAO-1 3-302. 

30 As  mentioned  previously  “major  construction  projects”  are  those  estimated  to  cost  more 
than  $10  million.  Of  VA’s  26  major  construction  projects,  23  are  estimated  to  cost  $100 
million  or  more. 

31To  assess  the  extent  to  which  cost  increases  and  schedule  delays  have  continued  at 
VA’s  major  construction  projects,  we  analyzed  how  estimated  costs  and  completion  dates 
for  ongoing  projects  have  changed  since  the  time  of  our  2013  report.  For  that  report,  VA 
provided  us  with  data  on  estimated  costs  and  completion  dates  for  its  50  ongoing  major 
medical-facility  projects  that  were  current  as  of  November  2012.  For  our  current  review, 
we  analyzed  how  estimated  costs  and  completion  dates  for  projects  that  are  still  ongoing 
have  changed  since  November  2012.  See  GAO-13-302. 

op 

We  did  not  calculate  changes  in  costs  for  two  projects — the  Manhattan  flood  recovery 
and  American  Lake  projects —  for  this  report.  The  Manhattan  project  had  no  cost  estimate 
available  in  November  2012.  The  American  Lake  project’s  scope  was  expanded  in  fiscal 
year  2015  to  include  additional  work,  an  expansion  that  increased  the  total  estimated  cost 
and  made  the  cost  in  November  2012  inapplicable. 

33We  did  not  calculate  changes  in  schedules  for  13  projects.  Ten  projects  are  pending 
acceptance  by  USACE  and  do  not  have  current  estimated  completion  dates,  and  three 
projects  did  not  have  schedule  data  published  in  November  2012. 
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Table  2:  Changes  in  Costs  and  Completion  Time  Frames  between  November  2012  and  October  2016  for  Selected  Department 
of  Veterans  Affairs’  (VA)  Medical-Facility  Construction  Projects 


Project 

location 

Estimated  cost, 
Nov.  2012 
(dollars  in 
thousands) 

Estimated  cost, 
Oct.  2016 
(dollars  in 
thousands) 

Percentage  (%) 
change 

Estimated 
completion 
timeframe,  Nov. 
2012 

Estimated 
completion 
timeframe,  Oct. 
2016 

Number  of 

months 

difference 

Denver 

$800,000 

$1,675,000 

109.4% 

April  2015 

Jan.  2018 

33 

Louisville3 

900,000 

925,000 

2.8 

NA 

NA 

NA 

New 

Orleans 

995,000 

1,084,500 

9.0 

Feb.  2016 

Dec.  2017 

22 

Palo  Altob 

716,600 

716,600 

0 

Dec.  2017 

June  2019 

18 

St.  Louis0 

366,500 

366,500 

0 

NA 

Aug. 2020 

NA 

Legend: 

NA=Not  available 

Source:  GAO  analysis  of  VA  data.  |  GAO-17-70 

aThe  Louisville  project  did  not  have  estimated  completion  dates  available  in  November  2012  or 
October  2016. 


bAccording  to  VA,  it  expects  the  cost  estimate  for  the  Palo  Alto  project  to  increase  and  is  evaluating 
options  to  offset  these  increases.  Additionally,  the  June  2019  completion  date  depends  on  VA’s 
receiving  construction  funding  in  2018. 

cThe  St.  Louis  project  did  not  have  an  estimated  completion  date  available  in  November  2012. 
Additionally,  according  to  VA,  the  August  2020  completion  date  depends  on  VA’s  receiving 
Congressional  funding  for  construction  in  2018. 


VA  Lacks  Sufficient  Data 
to  Analyze  the  Processing 
Time  Frames  and 
Reasons  for  Change 
Orders 


VA  has  taken  steps  to  improve  its  process  for  managing  change  orders. 
As  discussed,  we  found  in  2013  that  delays  in  VA’s  change-order 
approval  process  affected  costs  and  schedules  of  VA  projects.34  VA  has 
taken  several  actions  since  then  to  streamline  its  change-order  approval 
process,  including  establishing  processing  timeframes  and  increasing  the 
dollar  value  of  change  orders  that  need  VA  headquarters  approval. 


As  described  earlier,  VA  issued  its  Contract  Modification  Handbook  (the 
Handbook)  in  August  2013  to  establish  milestones  for  processing  change 
orders.  These  milestones  include  both  interim  milestones  throughout  the 
change  order  process  and  milestones  for  the  total  amount  of  time  a 
change  order  should  take  to  be  processed.  The  Handbook  specifies  time 
frames  for  change  orders  based  on  their  cost  (i.e.,  more  costly  change 
orders  have  longer  time  frames)  and  also  requires  VA  staff  to  enter  the 
prescribed  milestones  into  VA’s  Electronic  Contract  Management  System 


34GAO- 13-302. 
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(eCMS).  While  VA  officials  said  they  believe  that  processing  time  frames 
have  improved,  contractor  officials  at  one  site  we  visited  said  they  had  not 
seen  significant  improvements  in  the  process  and  cited  time  frames  as  a 
continuing  challenge  for  VA. 

However,  VA  currently  does  not  collect  the  necessary  information  to 
determine  whether  efforts  to  streamline  the  change  order  process  have  in 
fact  been  successful.  More  specifically,  eCMS  does  not  track  whether 
many  of  the  milestone  dates  are  met  as  required  by  the  Handbook.  VA 
provided  us  with  data  from  eCMS  on  change  orders  for  2014,  2015,  and 
2016.  We  found  that  this  data  contains  the  dates  when  change  orders 
were  created  in  the  system,  and  the  dates  when  they  finished  being 
processed.  However,  the  data  do  not  contain  information  on  all  of  the 
milestones  required  by  the  Handbook,  and  because  of  this  omission,  VA 
cannot  determine  if  its  guidelines  are  being  met.  Although  VA  officials 
said  they  do  periodically  review  eCMS  data,  because  they  contain  limited 
information  on  milestones  in  the  change  order  process,  this  data  cannot 
be  compared  to  the  Handbook’s  milestones.  CFM  officials  at  VA 
headquarters  said  that  eCMS  is  meant  to  generate  contract  actions  and 
was  not  designed  to  help  manage  construction  projects.  It  is  thus  not 
configured  to  collect  data  on  the  milestones  required  by  the  Handbook.35 

While  spreadsheets  containing  data  on  change  orders  are  prepared  at 
some  of  the  VA  sites  we  visited,  the  information  they  contain  on 
processing  time  frames  is  insufficient  to  assess  the  timeliness  of  change 
orders  against  requirements  in  the  Handbook.  Three  sites  we  visited  kept 
spreadsheets  on  change  orders  that  contained  information  on  processing 
time  frames,  including  some  of  the  time  frames  required  by  the 
Handbook.36  VA  headquarters  officials  told  us  that  regional  CFM  offices 
monitor  change-order-processing  time  frames  for  projects  in  their  regions 
using  these  spreadsheets.  However,  VA  officials  said  that  the  monitoring 
process  is  manual  and  not  done  for  all  change  orders.  Additionally,  the 
information  in  these  spreadsheets  was  incomplete  and  inconsistent.  For 
example,  these  spreadsheets  contained  incomplete  information  for  many 


35We  recently  reported  on  problems  with  eCMS’s  reporting.  See  GAO,  Veterans  Affairs 
Contracting:  Improvements  in  Policies  and  Processes  Could  Yield  Cost  Savings  and 
Efficiency ,  GAO-16-810  (Washington,  D.C.,  September,  2016). 

36These  three  projects  were  New  Orleans,  LA;  Palo  Alto,  CA;  and  St.  Louis,  MO.  We  did 
not  review  change  order  data  at  the  Denver,  CO,  project  because  USACE  manages 
change  orders  for  that  project.  The  Louisville,  KY,  project  currently  has  no  change  orders 
because  construction  on  that  facility  has  not  yet  started. 
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change  orders,  and  none  of  the  spreadsheets  contained  information 
sufficient  to  determine  whether  processing  time  frames  established  in  the 
Handbook  were  met.  Additionally,  each  site  we  visited  collected  different 
types  of  data  on  change  orders. 

Because  VA  lacks  the  data  on  the  change  order  processing  timeframes 
required  by  the  Handbook  or  similar  data  indicating  how  long  change 
orders  take  to  process,  it  is  unable  to  clearly  measure  the  impact,  if  any, 
of  its  actions  to  improve  the  change  order  process.  Using  complete  and 
consistent  information  to  monitor  its  adherence  to  required  time  frames  is 
important  for  VA  to  achieve  its  goals  of  processing  change  orders  in  a 
timelier  manner,  especially  given  our  previous  findings  that  change-order- 
processing  time  frames  caused  delays  at  some  projects.  However, 
because  VA  is  not  collecting  the  information  necessary  to  know  if  its 
guidelines  are  being  followed,  it  cannot  currently  measure  its  performance 
to  determine  if  its  improvement  actions  and  guidelines  have  succeeded  in 
improving  the  timeliness  of  processing  change  orders  or  whether 
additional  actions  are  needed. 

Similarly,  VA  does  not  collect  sufficient  information  to  monitor  the  reasons 
change  orders  occur.  Change  orders  can  occur  for  a  variety  of  reasons, 
including  errors  in  the  project’s  design,  unknown  conditions  at  the  project 
site  (such  as  buried  fuel  storage  tanks),  or  because  medical  center  staff 
request  a  change  to  the  project.  Although  VA  collects  some  information 
on  change  orders — such  as  the  dates  change  orders  were  approved  and 
their  amounts,  as  discussed  earlier — it  does  not  collect  sufficient  data  to 
categorize  and  monitor  the  reasons  for  change  orders.  While  a 
description  of  each  change  order  is  entered  into  eCMS,  this  information 
cannot  be  easily  categorized.  Additionally,  only  two  of  the  five  sites  we 
visited  collect  information  in  any  systemic  manner  related  to  the  reasons 
for  change  orders.37  However,  these  sites  do  not  do  so  consistently  or  for 
all  change  orders. 

VA  is  taking  steps  to  procure  a  new  system  to  more  consistently  collect 
information  on  change  orders.  As  of  December  2016,  VA  was  in  the 
process  of  procuring  a  new  system  that  could  better  collect  and  track 
information  on  change  orders  for  all  projects  and  provide  reports  to 
management.  According  to  VA’s  plans,  this  system  is  intended: 


37These  two  projects  were  New  Orleans,  LA,  and  Palo  Alto,  CA. 
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•  to  track  dates  associated  with  each  change  order  and  its  status, 

•  generate  change  order  alerts  based  on  timeframes, 

•  notify  VA  staff  when  they  are  expected  to  take  action  on  a  change 
order, 

•  track  the  reason  for  each  change  order,  and 

•  provide  reports  to  management,  among  other  things. 

According  to  the  Request  for  Proposal  for  the  new  system,  VA  was 
targeting  March  of  2017  to  have  the  new  system  in  place.  However,  in 
comments  on  a  draft  of  this  report,  VA  stated  that,  while  this  procurement 
has  been  delayed,  VA  has  developed  a  plan  to  monitor  compliance  with 
time  frames  for  approving  change  orders  established  in  the  Handbook, 
using  existing  reporting  structures.  VA  expects  to  have  this  system  in 
place  by  March  of  201 7. 

Although  VA’s  new  system  is  expected  to  collect  the  necessary 
information  on  change  orders,  it  is  not  clear  how  VA  plans  to  use  this 
information.  More  specifically,  VA  has  not  established  a  mechanism  to 
monitor  whether  change  orders  are  approved  within  the  guidelines  VA 
established  or  the  reasons  why  change  orders  occur.  In  September  2015, 
VA  issued  a  memo  identifying  timely  change  orders  as  a  goal.  Likewise, 
VA  issued  guidance  in  May  2016  directing  CFM  staff  to  minimize  changes 
the  VA  medical  centers  requested  during  construction  to  the  extent 
possible  because  they  can  be  disruptive  and  costly.38  An  effective 
internal-monitoring  mechanism  requires  VA  management  not  only  to 
collect  data  from  reliable  sources,  as  it  plans  to  do,  but  also  to  establish 
and  operate  monitoring  activities  that  include  using  collected  information 
on  change-order  and  design  change  controls  to  measure  performance 
toward  achieving  objectives.  According  to  the  Standards  for  Internal 
Control  in  the  Federal  Government,  ongoing  monitoring  should  be  built 
into  the  entity’s  operations,  performed  continually,  and  be  responsive  to 
change.  Further,  separate  evaluations  should  be  used  to  periodically 
provide  feedback  on  the  effectiveness  of  this  monitoring.  Without  a  fully 
functioning  monitoring  system,  VA  cannot  determine  the  extent  to  which 
processing  time  frames  and  design  changes  affect  project  costs  and 
schedules  and  thus  is  at  risk  of  unexpected  cost  increases  and  schedule 
delays  occurring  at  VA-managed  projects. 

38Department  of  Veterans  Affairs,  Office  of  Construction  and  Facilities  Management, 
Memorandum:  Management  of  User  Requested  Changes  in  Major  Construction  Projects 
(May  5,  2016). 
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The  Cost  Estimate  for  the 
Denver  Project’s 
Construction  Is  Reliable, 
but  Other  Estimates  Are 
Not 


As  part  of  our  review,  we  assessed  the  Denver  project’s  cost  and 
schedule  estimates  against  best  practices  for  estimating  costs  and 
schedules.  This  project  is  the  only  USACE-managed  project  that  is 
currently  under  construction  and  has  cost  and  schedule  data  available.  It 
involves  constructing  a  new,  1 . 2-million  square  foot  facility  that  includes 
148  patient  beds,  inpatient  tertiary  care  and  ambulatory  care,  a  30-bed 
spinal  cord  injury  center,  a  research  building,  parking  structures,  and 
many  supporting  facilities.  As  of  August  2015,  when  USACE  took  over 
management  of  the  project,  USACE  estimated  that  completing  the 
construction  of  the  facility  would  cost  an  additional  $585  million.  This 
estimate,  along  with  various  contract  oversight  and  USACE  management 
fees  of  $40  million,  $150  million  in  reallocated  VA  funding,  and  the  $900 
million  VA  has  already  spent  on  the  project,  brings  the  total  estimated 
cost  to  complete  construction  to  about  $1,675  billion — an  increase  of 
$1 .054  billion  over  the  initial  estimate.  Additionally,  as  of  May  201 6, 
USACE  estimates  that  the  project  will  be  completed  in  January  2018 — 
about  40  months  behind  VA’s  originally  scheduled  completion  date.  Our 
analysis  focused  on  the  estimated  cost  of  $585  million  to  complete 
construction  and  its  estimated  completion  date  of  January  2018.  These 
two  were  the  most  recent  estimates  available  at  the  time  of  our  review.39 
VA  is  responsible  for  activating  the  Denver  facility  and  has  estimated  that 
this  process  will  cost  an  additional  $341  million  and  will  be  completed  by 
June  2019. 


39This  analysis  focused  only  on  the  cost  and  schedule  estimates  for  the  contractor  to 
complete  construction  of  the  facility.  We  discuss  costs  and  schedules  related  to  activation 
of  the  facility  later  in  this  report. 
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The  August  2015  Cost 
Estimate  for  Construction  of 
the  Denver  Project 
Substantially  Meets  the  Four 
Characteristics  of  Reliable 
Cost  Estimates 


The  August  2015  cost  estimate  for  completing  construction  of  the  Denver 
project  substantially  meets  the  characteristics  of  reliable  cost  estimates 
identified  in  the  GAO  Cost  Estimating  and  Assessment  Guide.40  This 
guide  defines  best  practices  related  to  the  four  characteristics — 
comprehensive,  well  documented,  accurate,  and  credible — of  high- 
quality,  reliable  cost  estimates.  We  compared  the  Denver  construction 
cost  estimate  to  these  four  characteristics  and  used  five  categories — fully 
meets,  substantially  meets,  partially  meets,  minimally  meets,  or  does  not 
meet — to  rate  how  the  cost  estimate  met  each  characteristic.  We  consider 
cost  estimates  to  be  reliable  if  we  determine  that  the  overall  assessment 
ratings  for  each  of  the  four  characteristics  are  substantially  or  fully  met. 
Conversely,  if  our  analysis  showed  that  any  of  the  characteristics  were 
not  met,  minimally  met,  or  partially  met,  then  the  estimate  cannot  be 
considered  reliable.  We  found  that  the  construction  cost  estimate  for  the 
Denver  project  substantially  meets  each  of  the  four  characteristics  of  a 
high-quality,  reliable  cost  estimate  (see  table  3).  Appendix  IV  provides 
greater  detail  of  our  comparison  of  the  estimate  with  the  specific  best 
practices  that  constitute  these  characteristics. 


Table  3:  GAO’s  Assessment  of  the  Cost  Estimate  to  Complete  Construction  of  the  Department  of  Veterans  Affairs  Denver 
Medical  Center  Compared  to  the  Four  Characteristics  of  High-Quality,  Reliable  Cost  Estimates 


Characteristic 

Description  of  characteristic 

GAO  assessment 

Comprehensive 

•  The  estimate  accounts  for  all  possible  costs  associated  with  a  project. 

•  Estimate  is  structured  in  sufficient  detail  to  ensure  that  costs  are  neither  omitted 
nor  double  counted. 

•  Estimate  documents  all  cost-influencing  assumptions. 

Substantially  meets 

Well-documented 

•  Supporting  documentation  explains  the  process,  sources,  and  methods  used  to 
create  the  estimate. 

•  Estimate  contains  the  underlying  data  used  to  develop  the  estimate. 

•  Estimate  has  been  adequately  reviewed  and  approved  by  management. 

Substantially  meets 

40GAO-09-3SP.  We  determined  that  most  of  the  leading  practices  were  applicable  to  the 
assessment  of  the  cost  estimate  for  completing  construction  of  the  Denver  project. 
However,  we  determined  that  the  best  practice  of  having  a  group  outside  the  agency 
conduct  an  independent  cost  estimate  was  not  applicable  to  the  construction  cost  estimate 
because  the  estimate  itself  served  as  an  independent  cost  estimate.  The  purpose  of  the 
estimating  effort  was  to  develop  an  independent  cost  estimate  that  would  enable  the 
USACE  to  establish  a  firm  target  price  with  the  construction  contractor  for  the  remaining 
construction.  USACE’s  cost  estimate  served  as  an  independent  cost  estimate  for 
comparison  with  the  construction  contractor’s  estimate,  so  the  criteria  for  an  independent 
cost  estimate  is  inapplicable. 
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Characteristic 

Description  of  characteristic 

GAO  assessment 

Accurate 

•  The  estimate  is  not  overly  conservative  or  optimistic. 

•  Estimate  is  based  on  an  assessment  of  the  costs  most  likely  to  be  incurred. 

•  Estimate  is  regularly  updated  so  that  it  always  reflects  the  project’s  current  status. 

Substantially  meets 

Credible 

•  Any  limitations  of  the  analysis  because  of  uncertainty  or  sensitivity  surrounding 
data  or  assumptions  are  discussed. 

•  The  estimate’s  results  are  cross-checked. 

Substantially  meets 

Legend: 

Substantially  meets= 
criteria 

Source:  GAO,  |  GAO-17-70 

the  Department  of  Veterans  Affairs  and  the  U.S.  Army  Corps  of  Engineers  provided  evidence  that  satisfies  a  large  portion  of  the 

Notes:  For  the  purposes  of  this  analysis,  we  assessed  only  the  estimated  cost  for  the  contractor  to 
complete  construction  of  the  facility. 

Comprehensive 

A  cost  estimate  is  considered  comprehensive  if  it  accounts  for  all  possible 
costs  associated  with  a  project,  is  structured  in  sufficient  detail  to  ensure 
that  costs  are  neither  omitted  nor  double  counted,  and  documents  all 
cost-influencing  assumptions.  We  found  the  estimate  substantially  meets 
this  characteristic.  USACE’s  construction  cost  estimate  includes  all 
project  costs  of  the  remaining  construction.  USACE’s  description  of 
remaining  construction  work  was  based  on  an  extensive  review  of  the 
project’s  progress  through  April  2015  and  comprises  the  technical 
baseline  of  the  remaining  construction  work.  The  project’s  work 
breakdown  structure — which  is  intended  to  define  in  detail  the  work 
necessary  to  accomplish  a  project’s  objectives — is  based  on  end  products 
and  deliverables,  such  as  buildings,  and  has  an  appropriate  level  of  detail 
for  all  elements.  The  cost  estimate’s  documentation  also  addresses  the 
ground  rules  and  assumptions  on  which  the  estimate  was  based  and 
identifies  major  risks  to  the  project,  such  as  schedule  slippage.  However, 
the  estimate  does  not  link  these  risks  to  specific  work  breakdown 
structure  elements. 

Well  documented 

A  cost  estimate  is  considered  well  documented:  (1)  when  supporting 
documentation  explains  the  process,  sources,  and  methods  used  to 
create  the  estimate,  (2)  when  the  estimate  contains  the  underlying  data 
used  to  develop  the  estimate,  and  (3)  when  it  is  adequately  reviewed  and 
approved  by  management.  We  found  the  estimate  substantially  meets 
this  characteristic.  USACE’s  cost  estimate  data  were  collected  from 
primary  sources,  such  as  the  construction  plans  and  contractor  price 
quotes.  USACE’s  documentation  describes  the  methodology  and  includes 
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detailed  costs  for  each  cost  element.  This  documentation  also  includes 
detailed  information  about  the  work  breakdown’s  structure,  assumptions, 
and  exclusions,  but  does  not  demonstrate  the  step-by-step  calculations  or 
the  source  data  used  to  develop  each  element  of  the  estimate.  While 
officials  said  that  there  were  many  formal  briefings  to  upper  management, 
they  did  not  provide  documentation  of  these  briefings. 

Accurate 

A  cost  estimate  is  considered  accurate  when  the  estimate  is  not  overly 
conservative  or  optimistic,  is  based  on  an  assessment  of  the  costs  most 
likely  to  be  incurred,  and  is  regularly  updated  so  that  it  always  reflects  the 
project’s  current  status.  We  found  the  estimate  substantially  meets  this 
characteristic.  USACE  officials  said  that  all  of  the  cost  estimates  were 
double  checked  for  errors — we  used  spot  checks  and  found  no  errors. 
Additionally,  when  changes  had  to  be  made  to  the  estimate,  the  variations 
were  explained  in  detail  in  the  estimate’s  documentation.  Estimate 
updates  occurred  in  order  to  reflect  changes  due  to  continuing 
construction  and  better  understanding  of  project  costs.  Officials  said  that 
they  had  adjusted  the  estimate  for  inflation  and  provided  the  inflation 
indexes  used,  although  we  could  not  independently  verify  the  application 
of  those  indexes  because  the  estimate  calculations  were  not  provided. 
The  estimate  also  lacks  a  documented  comparison  of  the  current 
estimate  (updated  with  actual  costs)  and  previous  estimate.  However,  the 
estimate  is  based  on  historical  and  relevant  data  taken  from  the  existing 
contract,  remaining  construction  work,  and  cost  information  from  other 
hospitals. 

Credible 

A  cost  estimate  is  considered  credible  when  any  limitations  of  the 
analysis  are  discussed  and  the  estimate’s  results  are  cross-checked.  We 
found  USACE’s  estimate  substantially  meets  this  characteristic.  A  formal 
cost  risk  and  uncertainty  analysis  was  performed,  and  it  was  used  to  set 
the  amount  needed  for  cost  contingency  for  the  program.41  While  the 
estimate  does  not  include  a  sensitivity  analysis,  it  does  identify  key 


41 

Contingency  represents  funds  held  for  “unknown  unknowns”  that  are  outside  a 
construction  contractor’s  control. 
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The  Cost  Estimate  for 
Activating  the  Denver  Facility 
Is  Unreliable 


factors  affecting  cost  and  risk  and  examines  the  sensitivity  of  major  risks 
based  on  the  results  of  the  risk  and  uncertainty  analysis.42 

VA  is  required  to  include  activation  costs  in  the  estimated  cost  of  a  new 
medical  facility.  Specifically,  the  prospectus  that  VA  is  required  to  submit 
to  the  House  and  Senate  Committees  on  Veterans’  Affairs  for  a  major- 
medical  facility  project  must  include,  among  other  things,  a  detailed 
estimate  including  the  facility’s  estimated  activation  cost.43  Similarly, 
Office  of  Management  and  Budget  Circular  A-1 1  states  that  the  cost  of  a 
capital  asset  such  as  a  new  hospital  is  its  full  life-cycle  cost,  which 
consists  of  all  direct  and  indirect  costs  for  planning  through  disposal, 
including  activation.44  Finally,  the  GAO  Cost  Estimating  and  Assessment 
Guide  states  that  a  project’s  cost  estimate  should  include  both  the 
government’s  and  contractor’s  costs  of  the  project  over  its  full  life  cycle, 
which  includes  everything  from  the  project’s  inception  to  its  disposal.45 

VA  currently  estimates  that  activating  the  Denver  facility  will  cost  $341 
million,  but  the  estimate  lacks  documentation  supporting  this  figure.46 
VHA  officials  told  us  that  the  original  activation  estimate  of  $272  million 
was  developed  in  2012  using  a  predecessor  to  VA’s  current  model  for 
estimating  activation  costs,  called  the  Activation  Cost  Budget  Model 
(ACBM),  when  construction  of  the  facility  was  in  the  early  stages. 
According  to  VA  officials,  the  ACBM  is  being  revised  to  help  the  VA 
medical  center’s  staff  develop  more  accurate  activation  cost  estimates 
that  align  with  projects.  The  officials  said  that  since  2012,  the  estimate 
has  increased  to  $341  million  due  to  updated  information  such  as  more 


42A  sensitivity  analysis  can  help  estimators  identify  the  cost  elements  that  represent  the 
most  risk. 

4338  U.S.C.  §  8104(b)(1)(B). 

440ffice  of  Management  and  Budget  (OMB),  Capital  Programming  Guide,  Supplement  to 
OMB  Circular  No.  A-11,  “Appendix  I”  (2015). 

45GAO-09-3SP. 

46 According  to  VA  officials,  this  activation  estimate  also  includes  the  cost  of  continuing  to 
operate  a  Community  Living  Center  at  the  current  Denver  facility  after  the  new  facility 
opens.  Although  this  center  had  been  a  part  of  the  Denver  project’s  plan,  it  was  cut  from 
the  project  to  reduce  costs.  As  a  result,  VA  officials  said  that  the  current  facility  will  in  part 
need  to  remain  open  to  operate  this  center. 
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precise  equipment  requirements.47  However,  VA’s  $341 -million  cost 
estimate  for  activating  the  Denver  facility  is  not  well  supported. 
Specifically,  VA  officials  provided  minimal  supporting  documentation 
associated  with  developing  the  2012  cost  estimate,  on  which  the  current 
estimate  is  based.  Additionally,  VA  officials  said  that  the  VHA  staff 
responsible  for  it  are  no  longer  with  VA.  Current  Denver  VHA  officials  we 
interviewed  regarding  the  estimate  said  that  they  were  not  familiar  with  or 
involved  in  its  development.  These  officials  said  that  they  do  not  know 
why  documentation  on  developing  the  estimate  did  not  exist. 

Without  documentary  evidence  on  the  cost-estimating  process  for  the 
activation  estimate,  we  cannot  determine  the  extent  to  which  it  conforms 
to  the  characteristics  of  a  high-quality  cost  estimate  described  in  the  GAO 
Cost  Estimating  and  Assessment  Guide.48  As  a  result,  we  consider  the 
estimate  unreliable.  A  reliable  cost  estimate  is  critical  to  the  success  of 
any  program.  Such  an  estimate  provides  the  basis  for  informed  decision 
making,  realistic  budget  formulation  and  program  resourcing,  and 
accountability  for  results.  For  example,  VA  relies  on  these  estimates  to 
make  annual  funding  decisions  for  activating  various  facilities. 

Additionally,  because  these  estimates  inform  VA’s  annual  budget 
requests,  Congress  relies  on  them  to  make  annual  appropriations 
decisions.  According  to  VA  officials,  VA’s  Activations  Office  will  assist 
sites  with  estimating  activation  costs.  However,  VA  officials  said  that  the 
Activations  Office  had  not  provided  assistance  for  the  Denver  project. 
Without  a  reliable  cost  estimate  for  activating  this  project,  VA  and 
congressional  decision  makers  cannot  make  informed  decisions  such  as 
budgeting  the  funds  needed  to  fully  equip  the  Denver  VA  medical  center 
so  that  it  is  fully  operational. 


47VA  headquarters  officials  said  that  this  estimate  consists  of  one-time  costs,  although 
these  officials  said  that  they  provided  Congress  with  an  activation  estimate  in  2016  that 
also  included  recurring  costs  of  about  $55  million,  which  brings  the  total  operating  cost  of 
the  project  to  $396  million.  These  officials  told  us  that  they  did  not  consider  recurring  costs 
to  be  activation  costs.  However,  VA’s  guidance  on  activations  describes  recurring 
activation  costs  as  the  incremental  workload  resulting  from  a  project.  In  addition  to  this 
estimate,  VA  officials  at  the  Denver  project  told  us  that  the  current  activation  estimate  is 
$386  million,  and  these  officials  did  not  distinguish  between  one-time  costs  and  recurring 
costs. 

48GAO-09-3SP. 
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The  Denver  Construction 
Schedule  Only  Partially  Meets 
Most  Characteristics  of 
Reliable  Schedules,  Making 
the  Schedule  Unreliable 


The  most  recent  schedule  estimate — prepared  in  May  2016  by  USACE — 
for  completing  construction  of  the  Denver  project  is  also  not  reliable, 
based  on  our  criteria.  Specifically,  the  estimate  partially  meets  most 
characteristics  of  reliable  schedule  estimates  identified  in  the  GAO 
Schedule  Assessment  Guide.49  This  guide  defines  best  practices  related 
to  the  four  characteristics — comprehensive,  well-constructed,  credible, 
and  controlled — of  high-quality,  reliable  schedule  estimates.  A  project’s 
success  depends  in  part  on  having  a  reliable  schedule  describing  the 
project’s  work  activities,  the  resources  required  to  complete  them,  and 
how  they  relate  to  one  another.  Thus,  the  schedule  not  only  provides  a 
road  map  for  the  systematic  execution  of  a  program,  but  also  provides  a 
means  by  which  to  gauge  progress,  identify  and  address  potential 
problems,  and  promote  accountability.  We  compared  the  construction 
schedule  estimate  with  these  four  characteristics  and  used  five 
categories — fully  meets,  substantially  meets,  partially  meets,  minimally 
meets,  or  does  not  meet — to  rate  how  the  schedule  estimate  met  each 
characteristic.  We  consider  schedule  estimates  to  be  reliable  if  the  overall 
assessment  ratings  for  each  of  the  four  characteristics  are  substantially  or 
fully  met.  Conversely,  if  our  analysis  showed  that  any  of  the 
characteristics  were  not  met,  minimally  met,  or  partially  met,  then  the 
estimate  cannot  be  considered  reliable. 

We  found  that  the  construction  schedule  estimate  for  the  Denver  project 
substantially  meets  one  characteristic  of  high-quality,  reliable  schedule 
estimates  but  only  partially  meets  the  other  three  (see  table  4). 
Specifically,  the  schedule  substantially  meets  the  characteristic  of  a 
controlled  schedule.  By  only  partially  conforming  to  three  of  these 
characteristics,  VA  and  USACE  cannot  be  sure  that  the  schedule’s 
estimated  dates  are  reliable.  Appendix  V  provides  greater  detail  of  our 
comparison  of  the  estimate  with  the  specific  best  practices  that  constitute 
these  characteristics. 


49GAO-16-89G. 
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Table  4:  GAO  Assessment  of  Schedule  Estimate  to  Complete  Construction  of  the  Department  of  Veterans  Affairs’  (VA)  Denver 
Medical  Center  Compared  to  the  Four  Characteristics  of  High-Quality,  Reliable  Schedule  Estimates 


Characteristic  Description  GAO 

assessment 


Comprehensive 

•  The  schedule  estimate  includes  all  activities  for  both  the  government  and  its  contractors 
necessary  to  accomplish  a  project’s  objectives  as  defined  in  the  project’s  work 
breakdown  structure.3 

•  Schedule  estimate  includes  the  labor,  materials,  travel,  facilities,  and  equipment  needed 
to  do  the  work  and  depicts  when  those  resources  are  needed  and  when  they  will  be 
available. 

•  Schedule  estimate  realistically  reflects  how  long  each  activity  will  take  and  allows  for 
discrete  progress  measurement. 

Partially  meets 

Well-constructed 

•  Activities  in  the  schedule  estimate  are  logically  sequenced  with  the  most  straightforward 
logic  possible.  Unusual  or  complicated  logic  techniques  are  used  judiciously  and 
justified  in  the  schedule  documentation. 

•  The  schedule’s  critical  pathb  represents  a  true  model  of  the  activities  that  drive  the 
project’s  earliest  completion  date,  and  total  float3  accurately  depicts  schedule  flexibility. 

Partially  meets 

Credible 

•  The  schedule  estimate  is  horizontally  traceable — that  is,  it  reflects  the  order  of  events 
necessary  to  achieve  aggregated  products  or  outcomes. 

•  It  is  also  vertically  traceable — that  is,  activities  in  varying  levels  of  the  schedule  map  to 
one  another  and  key  dates  presented  to  management  in  periodic  briefings  are  in  sync 
with  the  schedule. 

•  Data  about  risks  and  opportunities  are  used  to  predict  a  level  of  confidence  in  meeting 
the  project’s  completion  date. 

•  The  level  of  necessary  schedule  contingency d  and  high-priority  risks  and  opportunities 
are  identified  by  conducting  a  robust  schedule  risk  analysis. 

Partially  meets 

Controlled 

•  The  schedule  estimate  is  updated  periodically  by  trained  schedulers  using  actual 
progress  and  logic  to  realistically  forecast  dates  for  program  activities. 

•  It  is  compared  against  a  designated  baseline  schedule  to  measure,  monitor,  and  report 
the  project’s  progress.  The  baseline  schedule  is  accompanied  by  a  basis  document  that 
explains  the  overall  approach  to  the  project,  defines  ground  rules  and  assumptions,  and 
describes  the  unique  features  of  the  schedule. 

•  The  baseline  schedule  and  current  schedule  are  subject  to  a  process  that  governs  when 
and  how  technical  and  programmatic  changes  are  applied. 

Substantially 

meets 

Legend: 

Substantially  met=VA  and  the  U.S.  Army  Corps  of  Engineers  (USACE)  provided  evidence  that  satisfies  a  large  portion  of  the  criteria 
Partially  met=VA/USACE  provided  evidence  that  satisfies  about  half  of  the  criterion 

Source:  GAO.  |  GAO-17-70 

Note:  For  the  purposes  of  this  analysis,  we  assessed  only  the  schedule  estimated  for  the  contractor 
to  complete  construction  of  the  facility. 

aA  work  breakdown  structure  defines  in  detail  the  work  necessary  to  accomplish  a  project’s 
objectives. 

bA  critical  path  is  the  sequence  of  activities  that  represents  the  longest  path  from  the  project’s  start 
and  finish  dates. 

cFloat  is  the  amount  of  time  by  which  an  activity  can  be  delayed  before  the  delay  affects  the  project’s 
estimated  finish  date. 

dContingency  is  a  reserve  of  extra  time  to  account  for  known  and  quantified  risks  and  uncertainty. 
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Comprehensive 

A  schedule  estimate  is  comprehensive  if  it:  (1)  includes  all  activities,  (2) 
depicts  what  resources  are  needed  and  when  they  will  be  available,  and 
(3)  realistically  reflects  how  long  activities  will  take.  We  found  that  the 
estimate  partially  meets  this  characteristic.  The  estimate  includes  all  work 
necessary  to  complete  construction,  and  activity  durations  were 
reasonably  short,  meaningful,  and  allowed  for  discrete  progress 
measurement.  However,  activities  in  USACE’s  Denver  construction 
schedule  are  not  consistently  mapped  to  a  well-defined  work  breakdown 
structure.  Additionally,  activities  in  the  schedule  do  not  have  resources, 
such  as  labor  and  equipment,  assigned  to  them.  Doing  so  could  help 
ensure  that  resources  are  adequate  and  allow  for  their  effective 
management.  If  the  schedule  does  not  allow  for  insight  into  the  current  or 
projected  allocation  of  resources,  the  risk  of  delay  is  significantly 
increased. 

Wei  I -constructed 

A  schedule  estimate  is  well-constructed  if: 

•  its  activities  are  logically  sequenced  with  the  most  straightforward 
logic  possible; 

•  unusual  or  complicated  logic  techniques  are  used  judiciously; 

•  the  schedule’s  critical  or  longest  path  represents  a  true  model  of  the 
activities  that  drive  the  project’s  earliest  completion  date;  and 

•  total  float  accurately  depicts  schedule  flexibility.50 

We  found  that  USACE’s  construction  schedule  estimate  for  Denver 
partially  meets  this  characteristic.  Activities  in  the  schedule  are  logically 
sequenced  and  the  longest  path  to  completion  is  valid  and  not  driven  by 
lags  or  constraints.  However,  many  activities  in  the  schedule  appear  to 
have  an  unreasonable  amount  of  total  float.  Unreasonable  total  float 
estimates  indicate  logic  weaknesses  in  a  schedule  and  may  result  in 
inaccurate  calculations  of  project  completion  dates. 


50Total  float  is  the  amount  of  time  by  which  an  activity  can  be  delayed  before  the  delay 
affects  the  project’s  estimated  finish  date. 
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Credible 


A  schedule  estimate  is  credible  if:  (1)  it  is  horizontally  and  vertically 
traceable,  (2)  data  about  risks  and  opportunities  are  used  to  predict  a 
level  of  confidence  in  the  project’s  completion  date,  and  (3)  the  level  of 
necessary  schedule  contingency  and  high-priority  risks  and  opportunities 
are  identified  by  conducting  a  robust  analysis  of  schedule  and  risk.  We 
found  that  the  estimate  partially  meets  this  characteristic.  Lower  level 
activities  in  USACE’s  construction  schedule  for  Denver  are  consistent 
with  higher  level  activities  in  the  work  breakdown  structure.  However,  the 
schedule  logic  has  gaps  that  indicate  the  schedule  may  not  depict 
relationships  between  different  project  elements.  Also,  key  milestone 
dates  at  varying  levels  of  the  schedule  do  not  map  to  one  another  or  to 
dates  presented  to  management,  meaning  that  the  schedule  may  present 
different  information  to  different  audiences.  Furthermore,  documentation 
does  not  contain  key  details  of  the  schedule  risk  analysis  USACE 
conducted  or  describe  how  activity  durations  were  simulated  for  that 
analysis.  If  a  schedule  risk  analysis  is  not  properly  conducted, 
determining  the  likelihood  of  the  program’s  completion  date,  how  much 
contingency  is  needed,  or  the  activities  that  are  most  likely  to  delay  the 
project  is  more  difficult. 

Controlled 

A  schedule  is  controlled  if  it  is:  (1)  updated  periodically  by  trained 
schedulers  using  actual  progress  and  logic  to  realistically  forecast  dates 
for  program  activities,  (2)  is  compared  against  a  designated  baseline 
schedule,  (3)  defines  ground  rules  and  assumptions,  and  (4)  describes 
the  unique  features  of  the  schedule.  We  found  that  the  estimate 
substantially  meets  this  characteristic.  The  Denver  construction  schedule 
is  updated  regularly  and  was  current  at  the  time  of  our  review. 

Additionally,  officials  provided  us  with  a  valid  baseline  schedule  from 
October  2015.  The  baseline  schedule  and  current  schedule  are  subject  to 
a  process  that  governs  when  and  how  technical  and  programmatic 
changes  are  applied.  USACE  also  examines  the  schedule  after  each 
update.  However,  we  found  no  evidence  of  a  schedule  narrative  that 
includes  important  information  about  updates. 

USACE  officials  explained  they  would  follow  best  practices  if  they  initiated 
a  project.  However,  they  stated  that  this  project  presented  a  unique 
situation  because  USACE  began  managing  the  project  when  it  was  about 
50  percent  complete.  Consequently,  many  normal  requirements  were  not 
included  in  the  schedule.  For  example,  the  construction  contract  between 
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USACE  and  the  construction  contractor  did  not  require  all  of  the  features 
of  reliable  schedules,  including  that  the  schedule  have  resources,  such  as 
labor  and  materials,  assigned  to  activities  or  be  mapped  to  a  work 
breakdown  structure.  However,  USACE  normally  requires,  for  example, 
that  schedules  be  mapped  to  a  work  breakdown  structure.  USACE 
officials  further  explained  that  incorporating  all  best  practices  into  the 
Denver  construction  schedule  now  would  be  costly  and  disruptive.  VA 
officials  also  noted  that  as  of  November  2016,  the  project  was  ahead  of 
schedule.  In  commenting  on  a  draft  of  this  report,  VA  noted  that  it  is 
confident  the  construction  contractor  is  currently  meeting  or  exceeding 
scheduled  dates  to  turn  buildings  over  to  VA  so  that  VA  can  begin  the 
activation  process. 


VA’s  Policies  on  Linking 
Construction  and 
Activation  Activities  Are 
Not  Clear  or  Consistent 


Various  VA  policies  require  that  CFM  link  construction  and  activation 
schedules  to  form  an  integrated  master  schedule  (IMS) — an  important 
element  to  ensuring  the  successful  and  timely  completion  of  those 
projects.51  According  to  the  Schedule  Assessment  Guide,52  an  IMS  helps 
ensure  that  all  activities  needed  to  complete  a  project  are  accounted  for 
and  ordered  correctly.  An  IMS  ideally  takes  the  form  of  a  single  schedule 
file  that  includes  all  activities.  However,  it  may  also  be  a  set  of  separate 
schedules  representing  the  work  of  separate  contractors  and  government 
offices  networked  together.  Best  practices  for  developing  schedules  in  the 
GAO  Schedule  Assessment  Guide  state  that  projects  should  have  an 
IMS.  Furthermore,  these  best  practices  state  that  all  activities  in  the  IMS 
be  logically  sequenced,  with  steps  clearly  showing  how  related  portions  of 
work  depend  on  one  another.  Finally,  Standards  for  Internal  Control  in  the 
Federal  Government  emphasize  the  importance  of  control  activities — 
including  issuing  policies  and  procedures — and  internal  communication — 
including  providing  quality  information  to  key  staff — to  achieving  agencies’ 
objectives  and  addressing  related  risks.53 


51  Department  of  Veterans  Affairs,  Requirement  for  Integrated  Master  Schedules  and  Cost 
Risk  Analysis  (Mar.  30,  2012);  Department  of  Veterans  Affairs,  Office  of  Construction  and 
Facilities  Management  (CFM)  Policy  Memorandum  003C-2014-9:  Integrated  Master 
Schedule  for  Major  Construction  Projects  and  Real  Property  Leases  (VAIQ  7486874)  (Oct. 
3,  2014);  and  Department  of  Veterans  Affairs,  Integrated  Master  Schedule  for  Major 
Construction  Projects,  Schedule  Development  (Feb.  23,  2016). 

52GAO-16-89G. 

53GAO-14-704G. 
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Although  VA  and  USACE  officials  at  the  Denver  project  provided  a 
construction  schedule,  an  activation  schedule,  and  an  IMS,  we  found  that 
certain  activation  activities  and  milestones  in  these  schedules  were  not 
aligned  with  each  other  across  the  three  schedules.  This  lack  of 
alignment  could  increase  the  risk  of  VA’s  experiencing  delays  in  activating 
major  medical  facilities. 

•  Our  analysis  showed  that  not  all  of  the  milestone  dates  in  the  IMS 
were  aligned  with  dates  in  the  activation  and  construction  schedules. 
For  example,  the  date  on  which  VA  is  scheduled  to  accept  the 
Diagnostic  and  Treatment  building  from  the  construction  contractor  is 
different  in  the  three  schedules.  The  IMS  shows  this  date  as  January 
24,  2018;  the  construction  schedule  shows  it  as  October  20,  2017, 
and  the  activation  schedule  shows  it  as  October  31 , 2017. 

•  VHA  officials  told  us  that  they  last  updated  the  activation  schedule  in 
January  2016.  This  is  in  contrast  to  the  construction  schedule,  which 
had  been  updated  in  May  2016,  and  the  IMS,  which  had  been 
updated  in  April  2016.  Although  the  dates  these  schedules  were  last 
updated  were  not  aligned,  we  were  able  to  compare  the  acceptance 
milestones  for  the  different  buildings  at  the  Denver  site  in  each  of  the 
three  schedules.  While  the  IMS  and  activation  milestones  were 
aligned  on  four  dates  on  which  VA  would  accept  buildings,  none  of  the 
construction  acceptance  milestones  were  aligned  with  those  in  either 
the  IMS  or  activation  schedule. 

•  While  1 1  activation  activities  were  in  the  construction  schedule  for 
initial  outfitting  and  transitioning  to  completed  buildings,  these  were 
not  aligned  well  with  the  activation  schedule:  our  analysis  of  these 
activities  in  the  construction  schedule  indicated  that  they  were 
untraceable  to  dates  in  the  activation  schedule.  In  fact,  the  activation 
activities  in  the  construction  schedule  all  ended  on  the  construction 
contract’s  end  date  in  January  2018.  Because  of  this  lack  of 
alignment,  activation  milestones  in  the  construction  schedule 
appeared  to  only  represent  construction  contractor  activation  efforts 
and  not  those  of  VA  medical  center  staff,  who  are  responsible  for 
overall  activation  of  the  facility. 

In  addition,  because  the  activation  schedule  was  developed  several 
months  earlier  than  the  construction  schedule  and  the  IMS,  dates  for 
activation  activities  in  the  construction  schedule  and  IMS  may  not  be 
accurate.  VHA  officials  explained  that  they  had  let  their  contract  with  the 
vendor  that  produced  the  original  activation  schedule  expire  after  the 
schedule  was  updated  in  January  2016.  As  of  August  2016,  VA  had  hired 
another  scheduling  contractor  that  developed  a  new  activation  schedule. 
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At  that  time,  VHA  officials  stated  that  they  were  in  the  process  of 
incorporating  major  construction  milestones  into  this  schedule.  USACE 
officials  noted  that  they  had  provided  construction  dates  to  the  activation 
contractor  as  a  part  of  this  effort.  According  to  USACE  officials  at  the  site, 
VHA’s  new  activation  contractor  is  now  responsible  for  the  IMS.  VHA 
officials  said  that  the  new  activation  schedule  includes  all  activation 
activities  and  will  also  incorporate  major  construction  milestones. 
However,  USACE  officials  said  that  they  have  no  plans  to  fully  integrate 
the  new  activation  schedule  with  the  construction  schedule.  VA  noted  in 
its  comments  on  a  draft  of  this  report  that  it  is  working  to  synchronize  its 
current  activation  schedule  with  USACE’s  construction  schedule. 

Although  VA  requires  an  IMS,  many  of  its  policies  on  developing  an  IMS 
that  links  construction  and  activation  activities  are  not  clear  or  consistent. 
Various  policies  that  CFM  issued  from  2012  through  2016  require  the 
CFM  project  manager  to  develop  an  IMS  for  all  projects,54  gathering 
project  schedule  data  from  various  sources,  including  medical  centers. 
However,  these  policies  use  conflicting  and  undefined  terms  to  describe 
the  activities  an  IMS  should  cover.  In  particular,  while  some  policies 
specify  an  IMS  should  cover  activities  through  project  “closeout,”  they 
also  note  that  the  IMS  should  cover  all  significant  activities  throughout  the 
project’s  “life  cycle.”  These  two  terms  are  undefined  and  can  be 
interpreted  to  encompass  different  activities:  “life  cycle,”  which 
encompasses  activities  through  disposal  of  the  facility  at  the  end  of  its 
useful  life,  includes  activation,  but  “closeout”  may  not.  Although  CFM’s 
policies  are  unclear,  guidance  from  VA’s  Activations  Office  specifically 
requires  that  a  project’s  activation  schedule  be  linked  to  the  CFM 
construction  schedule.  Moreover,  neither  CFM’s  nor  the  Activations 
Office’s  guidance  on  IMS  are  aligned  with  individual  projects’  key  plans — 
specifically,  the  individual  project  management  plans  (PMP) — the 
framework  plans  for  the  successful  execution  of  individual  projects  such 
as  the  one  in  Denver.  For  example,  the  Denver  PMP  says  that  the  IMS 
and  activation  schedules  are  two  different  schedules  and  has  no 
requirement  to  link  the  two  schedules,  an  approach  that  conflicts  with 
some  VA  policies  that  say  that  the  IMS  should  include  both  construction 
and  activation  activities. 


54Department  of  Veterans  Affairs,  Requirement  for  Integrated  Master  Schedules  and  Cost 
Risk  Analysis',  Department  of  Veterans  Affairs,  Integrated  Master  Schedule  for  Major 
Construction  Projects  and  Real  Property  Leases',  and  Department  of  Veterans  Affairs, 
Integrated  Master  Schedule  for  Major  Construction  Projects,  Schedule  Development. 
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The  fact  that  VA  does  not  have  a  fully  integrated  IMS  could  result  in 
additional  delays  for  the  Denver  project.  For  example,  VHA  officials  at  the 
project  said  that  some  completed  buildings  will  only  be  accessible  through 
areas  still  under  construction,  meaning  they  will  be  unavailable  to  occupy. 
Similarly,  these  officials  said  that  the  Diagnostic  and  Treatment  Center, 
which  is  critical  to  hospital  operations,  will  be  completed  last.  According  to 
these  officials,  this  building  will  be  required  to  support  other  buildings, 
such  as  the  inpatient  clinic,  that  will  be  completed  earlier.  These  issues 
could  delay  opening  the  new  facility.  Veterans  Service  Organizations  we 
spoke  with  told  us  that  veterans  in  the  local  area  are  affected  the  most 
when  a  hospital  does  not  open  on  time.  Veterans  in  the  Denver  area  told 
us  that  they  must  continue  to  receive  treatment  at  an  outdated  and 
deteriorating  facility  while  they  wait  for  the  new  hospital  to  open. 


Conclusions 


In  response  to  GAO’s  2013  report  and  statutory  mandates,  VA  has  taken 
a  number  of  actions  in  recent  years  to  improve  its  management  of 
medical-facility  projects,  particularly  projects  costing  over  $100  million. 
These  actions  have  resulted  in  benefits,  including  fostering  a  more 
collaborative  environment  among  contractors  and  VA’s  construction  and 
medical  staff.  However,  as  our  review  shows,  VA  needs  to  take  further 
actions.  We  found  that  VA  does  not  collect  sufficient  information  to 
determine  if  its  new  guidelines,  intended  to  ensure  the  timely  processing 
of  change  orders  and  better  control  costly  design  changes  during 
construction,  are  being  followed,  due  primarily  to  limitations  in  VA’s 
electronic  system  for  tracking  changes  to  construction  contracts.  VA 
intends  to  implement  a  system  by  March  2017  that  better  tracks  change 
orders.  However,  VA  does  not  have  a  mechanism  in  place  to  evaluate  the 
new  data  it  will  collect  to  periodically  provide  feedback  on  the 
effectiveness  of  the  VA’s  strengthened  guidelines  for  change  orders. 

In  response  to  statutory  mandates,  VA’s  enlistment  of  USACE  to  manage 
certain  projects  costing  $100  million  or  more  represents  a  significant 
action.  USACE  is  now  managing  the  Denver  project,  which  experienced 
large  cost  increases  and  significant  delays  when  it  was  managed  by  VA. 
Awarding  a  new  contract  with  the  contractor  to  continue  construction 
under  USACE  management  was  a  positive  step.  However,  additional 
steps  are  required  to  manage  the  project  and  avoid  further  cost  increases 
and  delays.  Specifically,  the  current  estimate  of  activation  costs 
developed  by  VA  is  unreliable.  Without  a  reliable  activation  cost  estimate, 
VA  has  no  assurance  that  current  funding  will  be  sufficient  to  complete 
activation  and  Congress,  veterans  and  the  American  public  do  not  have  a 
complete  and  accurate  picture  of  the  total  cost  of  the  Denver  project. 
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Further,  our  analysis  of  the  new  construction  schedule  for  the  project 
indicates  that  USACE  and  VA  need  to  more  fully  follow  the  key 
components  of  a  reliable  schedule,  such  as  integrating  and  aligning 
construction  and  activation  activities  into  a  master  schedule.  Otherwise, 

VA  and  USACE  risk  delays  in  completing  construction  on  and  activating 
the  Denver  facility.  Any  such  delays  could  result  in  additional  cost 
increases  and  further  postponing  the  time  when  veterans  will  be  able  to 
receive  services  at  the  new  facility.  Moreover,  the  confusing  and 
inconsistent  language  in  VA’s  policies  on  developing  an  IMS  will  make  it 
difficult  for  VA  to  move  forward  and  create  integrated  and  accurate 
schedules  in  the  future. 

Recommendations  for 
Executive  Action 

To  improve  VA’s  management  of  medical-facility  construction  projects 
and  its  accountability  and  to  allow  for  more  informed  decision  making  by 
Congress  and  VA,  we  recommend  that  the  Secretary  of  Veterans  Affairs 
take  the  following  three  actions: 

•  establish  a  mechanism  to  monitor  the  extent  that  major  facilities 
projects  are  following  guidelines  on  change  orders’  time  frames  and 
design  changes; 

•  develop  an  activation  cost  estimate  for  the  Denver  project  that  is 
reliable  and  conforms  with  best  practices  as  described  in  the  GAO 

Cost  Estimating  and  Assessment  Guide ;  and 

•  clarify  CFM  policies  to  require  that:  (1)  all  projects  have  an  integrated 
master  schedule  to  ensure  that  the  integrated  master  schedules 
include  and  link  all  construction  and  activation  activities,  and  (2)  the 
policies  on  integrated  master  schedule  for  projects  managed  by  CFM 
and  USACE  are  consistent. 

Agency  Comments 

We  provided  a  draft  of  this  report  to  VA  and  USACE  for  their  review  and 
comment.  VA  concurred  with  our  recommendations  and  provided  updated 
information,  which  we  incorporated  as  appropriate.  Both  VA  and  USACE 
provided  technical  comments,  which  we  also  incorporated  as  appropriate. 
VA’s  comments  are  reprinted  in  appendix  VI. 

We  are  sending  copies  of  this  report  to  appropriate  congressional 
committees,  the  Secretary  of  Veterans  Affairs,  the  Secretary  of  Defense, 
and  the  Commanding  General  and  Chief  Engineer  of  the  U.S.  Army 

Corps  of  Engineers.  In  addition,  the  report  is  available  at  no  charge  on  the 
GAO  website  at  http://www.gao.gov. 

Page  33 


GAO-17-70  VA  Construction 


If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
me  at  (202)  512-2834  orwised@gao.gov.  Contact  points  for  our  Offices 
of  Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last 
page  of  this  report.  GAO  staff  who  made  key  contributions  to  this  report 
are  listed  in  appendix  VII. 


David  J.  Wise 

Director,  Physical  Infrastructure 
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Appendix  I:  Scope  and  Methodology 


To  determine  the  actions  the  Department  of  Veterans  Affairs  (VA)  has 
taken  to  address  challenges  in  the  administration  and  oversight  of 
medical-facility  construction  projects  since  2013,  we  reviewed  VA’s 
policies  and  guidance  on  project  administration  and  oversight.  We 
obtained  and  analyzed  data  that  VA  provided  on  the  status  of  VA’s  26 
active  major  medical  facility  projects  that  had  received  funding  as  of 
March  2016,  including  the  total  estimated  cost,  scheduled  completion 
date  and  the  project’s  current  status.1  We  determined  there  were  23 
projects  estimated  to  cost  over  $100  million  within  the  list  of  26  active 
major-medical  facility  projects.  Of  these  23  projects,  10  have  agreements 
in  place  between  VA  and  the  United  States  Army  Corps  of  Engineers 
(USACE)  for  USACE  to  manage  various  phases  of  the  projects  as  of 
November  2016.  There  are  also  agreements  in  place  for  two  additional 
projects  not  included  in  VA’s  data — Reno,  Nevada  and  Portland,  Oregon.2 
We  selected  5  of  these  23  construction  projects  for  in-depth  case  study 
review  in  order  to  select  from  projects  with  the  highest  cost,  projects  in 
different  phases  of  the  construction  process,  as  well  as  a  mix  of  projects 
managed  by  VA’s  Office  of  Construction  and  Facility  Management  (CFM) 
or  USACE.3  For  the  selected  projects,  we  visited  the  Denver  CO,  New 
Orleans  LA,  and  Palo  Alto  CA  sites  and  conducted  teleconferences  for 
the  Louisville  KY  and  St.  Louis  MO  projects.  The  information  from  our 
selected  projects  is  illustrative  and  cannot  be  generalized  to  sites  agency¬ 
wide.  For  each  project,  we  reviewed  construction  documents;  examined 
cost,  schedule,  and  change-order  data;  interviewed  CFM  and,  where 
applicable,  USACE  officials  responsible  for  managing  design  and 
construction,  design  and  construction  contractors,  VA’s  Veterans  Health 


^he  term  “major  medical  facility  project”  is  defined  to  mean  a  project  for  the  construction, 
alteration,  or  acquisition  of  a  medical  facility  involving  a  total  expenditure  of  more  than 
$10,000,000.  See  38  U.S.C.  §  8104(a)(3)(A). 

2As  the  23  active  construction  projects  costing  $100  million  or  more  are  in  various  stages 
of  design,  acquisition  and  construction,  VA  and  USACE  are  negotiating  what  project  stage 
will  be  completed  by  VA  or  assigned  to  USACE. 

VA  and  USACE  have  agreements  for  American  Lake,  WA;  Canandaigua,  NY;  Denver, 

CO;  Livermore,  CA;  Long  Beach  CA;  Louisville,  KY;  Portland  OR;  Reno,  NV;  San  Diego, 
CA;  San  Francisco,  CA;  Tampa,  FL;  and  West  Los  Angeles,  CA.  VA  and  USACE  have  not 
yet  reached  an  agreement  for  the  Alameda,  CA  project. 

3The  projects  we  selected  are  located  in  Denver,  CO;  Louisville,  KY;  New  Orleans,  LA; 
Palo  Alto,  CA;  and  St.  Louis,  MO.  Of  these,  Denver  and  Louisville  have  agreements  for 
USACE  to  manage  construction.  The  site  that  we  refer  to  throughout  this  report  as  the 
Denver  VA  Medical  Center,  or  Denver,  is  actually  located  in  Aurora,  Colorado,  near 
Denver. 
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Administration  (VHA)  medical  center  staff,  and  representatives  of  local 
veterans  services  organizations. 

To  determine  whether  VA  could  make  any  additional  improvements  to  its 
administration  and  oversight  of  those  projects  costing  $100  million  or 
more,  we  compared  the  management  of  our  selected  projects  to  VA’s 
policies  and  procedures,  particularly  those  put  in  place  since  our  2013 
report,  to  determine  the  extent  to  which  they  are  followed.  We  interviewed 
VA  headquarters  and  USACE  officials  on  project  administration  and 
oversight  and  changes  intended  to  improve  these  processes.  We 
reviewed  prior  GAO,  VA  Office  of  Inspector  General  and  USACE  reports 
on  VA’s  management  of  these  projects. 

We  also  collected  CFM  information  on  cost  increases  and  schedule 
changes  for  the  five  construction  projects.  Because  of  its  dramatic  project 
cost  increase  and  status  as  the  only  project  that  USACE  manages  that  is 
currently  under  construction,  we  selected  the  Denver  cost  estimate  and 
schedule  for  an  analysis  of  compliance  with  best  practices.  We  discussed 
the  Denver  activation  cost  and  schedule  with  VHA  local  and  headquarters 
staff,  as  these  items  were  not  integrated  within  the  construction  estimate 
and  schedule.  We  assessed  the  reliability  of  the  Denver  construction  cost 
and  schedule  data  through  interviews  with  knowledgeable  VA  staff  and  a 
review  for  completeness  and  any  unexpected  values.  We  determined  that 
the  data  were  sufficiently  reliable  for  the  purpose  of  our  reporting 
objectives. 

To  determine  the  extent  to  which  VA  and  USACE  have  estimated  the  cost 
and  schedule  of  the  Denver  project  in  a  manner  consistent  with  best 
practices,  we  interviewed  VA  and  USACE  staff  and  compared  the  project 
cost  and  schedule  estimates  with  GAO  best  practices.  Specifically,  the 
GAO  Cost  Estimating  and  Assessment  Guide  identifies  best  practices 
that  represent  work  across  the  federal  government  and  are  the  basis  for  a 
high-quality,  reliable  cost  estimate.4  A  cost  estimate  created  using  best 
practices  exhibits  four  broad  characteristics:  it  is  accurate,  well 
documented,  credible,  and  comprehensive.  That  is,  each  characteristic  is 
associated  with  a  specific  set  of  best  practices.  In  turn,  each  best  practice 
is  made  up  of  a  number  of  specific  tasks  (see  app.  IV).  Similarly,  we 
compared  the  schedule  estimate  with  the  GAO  Schedule  Assessment 


4 GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs  (Supersedes  GAO-07-1 134SPJ,  GAO-09-3SP 
(Washington,  D.C.:  Mar.  2,  2009). 
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Guide,  which  defines  best  practices  related  to  four  characteristics — 
comprehensive,  we  1 1 -constructed,  credible,  and  controlled — that  are 
important  to  developing  high-quality,  reliable  schedule  estimates  (see 
app.  V).5  For  our  evaluations  of  the  cost  and  schedule  estimates,  when 
the  tasks  associated  with  the  best  practices  that  define  a  characteristic 
were  mostly  or  completely  satisfied,  we  considered  the  characteristic  to 
be  substantially  or  fully  met.  When  all  four  characteristics  were  at  least 
substantially  met,  we  considered  a  cost  or  schedule  estimate  to  be 
reliable.6 

We  conducted  this  performance  audit  from  January  2016  to  March  2017 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


5  GAO  Schedule  Assessment  Guide:  Best  Practices  for  Project  Schedules,  GAO-16-89G 
(Washington,  D.C.:  Dec.  22,  2015).  The  GAO  Schedule  Assessment  Guide  presents  the 
scheduling  concepts  introduced  in  the  Cost  Estimating  and  Assessment  Guide  as  10 
leading  practices  associated  with  developing  and  maintaining  a  reliable,  high-quality 
schedule.  The  leading  practices  were  developed  in  conjunction  with  government  and 
industry  experts  in  the  schedule-estimating  community.  The  GAO  Schedule  Assessment 
Guide  serves  also  to  present  guiding  principles  for  our  auditors  in  evaluating  the  economy, 
efficiency,  and  effectiveness  of  government  programs. 

6We  established  five  descriptions  for  our  assessments  of  leading  practices  and  cost 
estimate  characteristics:  fully  meets,  substantially  meets,  partially  meets,  minimally  meets, 
and  does  not  meet.  We  consider  a  leading  practice  to  be  fully  met  when  the  associated 
tasks  are  completely  satisfied,  substantially  met  when  a  large  portion  of  the  associated 
tasks  are  satisfied,  partially  met  when  about  half  of  the  associated  tasks  are  satisfied, 
minimally  met  when  a  small  portion  of  the  associated  tasks  are  satisfied,  and  not  met 
when  none  of  the  associated  tasks  are  satisfied.  Our  assessment  method  weights  each 
leading  practice  equally  and  bases  the  assessment  of  each  characteristic  on  the  average 
score  of  underlying  leading  practices.  We  assign  each  description  a  numerical  value  (5  for 
fully  meets  to  1  fordoes  not  meet)  and  round  scores  to  the  higher  numerical  value  (i.e. ,  a 
score  of  4.5  would  round  up  to  5  and  a  score  of  4.4  would  round  down  to  4).  Assessments 
were  conducted  by  an  individual  analyst,  and  then  the  results  were  independently  traced 
and  verified  by  a  second  analyst. 
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Appendix  II:  Additional  Information  on 
Medical  Center  Projects  Outsourced  to  U.S. 
Army  Corps  of  Engineers 


Table  5:  List  of  Department  of  Veterans  Affairs  Medical  Center  Projects  Outsourced  to  the  U.S.  Army  Corps  of  Engineers 
(USACE)  for  Construction 

Project 

State 

Description 

Status 

American  Lake 

WA 

New  Building  201  and  Seismic  Corrections  to 

Buildings  81  and  18 

Construction  Documents 

Canandaigua 

NY 

New  Construction  and  Renovation 

Construction  Documents 

Denver 

CO 

New  Medical  Facility 

Construction 

Long  Beach 

CA 

Seismic  Corrections  to  Mental  Health  and  Community 
Living  Center 

Design  Development,  Construction 
Documents,  and  Construction 

Louisville 

KY 

New  Medical  Facility 

Design  Development 

Palo  Alto 

CA 

Livermore  Realignment 

Design  Development 

Portland 

OR 

Seismic  Retrofit  and  Renovation  of  Buildings  100  and 
101  and  Add  Specialty  Clinic  and  parking  for  600 

Planning 

Reno 

NV 

Upgrade  Building  1  Seismic,  Life  Safety,  Utility 
Corrections,  and  Expand  Clinical  Services 

Design  Development 

San  Diego 

CA 

Seismic  Corrections 

Construction  Documents 

San  Francisco 

CA 

Seismic  Corrections  to  Buildings  1, 6,  8  and  12 

Construction  Documents 

Tampa 

FL 

Polytrauma  Expansion/Bed  Tower 

Design  Development 

West  Los  Angeles 

CA 

Seismic  Corrections  of  Various  Buildings 

Construction  Development  and  Construction 

Source:  GAO  analysis  of  VA  data.  |  GAO-17-70 


Note:  VA  and  USACE  have  agreements  in  place  for  USACE  to  complete  for  these  projects  as  of 
October  2016. 
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To  assess  the  extent  to  which  cost  increases  and  schedule  delays  have 
continued  at  Department  of  Veterans  Affairs  (VA)  major  construction 
projects,  we  analyzed  how  estimated  costs  and  completion  dates  for 
ongoing  projects  have  changed  since  the  time  of  our  2013  report  (see 
table  6).1  For  that  report,  VA  provided  us  with  data  on  estimated  costs 
and  completion  dates  for  its  50  ongoing  major  medical-facility  projects 
that  were  current  as  of  November  2012.  For  our  current  review,  we 
analyzed  how  estimated  costs  and  completion  timeframes  for  projects 
that  are  still  ongoing  have  changed  between  November  2012  and 
October  2016. 


Table  6:  Changes  in  Costs  and  Completion  Time  Frames  for  Department  of  Veterans  Affairs’  (VA)  Major  Medical-Facility 
Projects  between  November  2012  and  October  2016 

Project 

Project  Description 

Estimated 
cost, 
Nov.  2012 

Estimated 
cost, 
Oct  2016 

Estimated 

Percent  completion 
(%)  timeframe, 
change3  Nov.  2012b 

Estimated 

completion 

timeframe, 

Oct.  201 6C 

Number  of 
months 
difference11 

VA-managed  projects 

Bay  Pines 

Inpatient/Outpatient 

Improvements 

$158,200,000 

$158,200,000 

0.0  February-2015 

February-2020 

60 

Biloxi 

Restoration  Of 
Hospital/Consolidation 
of  Gulfport 

304,000,000 

297,000,000 

-2.3  June-2016 

August-2018 

26 

Dallas 

Spinal  Cord  Injury 

155,200,000 

155,200,000 

0.0  December-2014 

January-2017e 

25 

Las  Vegas 

New  Medical  Facility 

584,655,000 

584,655,000 

0.0  June-2014 

February-2016 

20 

Long  Beach 

Seismic  Corrections  to 
Buildings  7  and  126 

129,545,000 

129,545,000 

0.0  August-2014 

February-2022' 

90 

New  Orleans 

New  Medical  Facility 

995,000,000 

1,084,500,000 

9.0  February-2016 

December-2017 

22 

New  York 

Manhattan  Flood 
Recovery 

NA9 

207,000,000 

NA  NA 

February-2019 

NA 

Orlando 

New  Medical  Facility 

616,158,000 

616,158,000 

0.0  July-2013 

October-2016 

39 

Palo  Alto 

Centers  for  Ambulatory 
Care/Polytrauma-Blind 
Rehabilitation 

716,600,000 

716,600,000 

0.0  December-2017 

June-2019h 

18 

Perry  Point 

Replacement 

Community  Living 

Center 

90,100,000 

92,700,000 

2.9  TBD 

June-2020 

NA 

San  Juan 

Seismic  Corrections  to 
Building  1 

277,000,000 

277,000,000 

0.0  October-2016 

June-2021 

56 

1  Major  projects  are  those  that  VA  estimates  will  cost  more  than  $10  million. 
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Project 

Project  Description 

Estimated 
cost, 
Nov.  2012 

Estimated 
cost, 
Oct  2016 

Estimated 

Percent  completion 
(%)  timeframe, 
change3  Nov.  2012b 

Estimated 

completion 

timeframe, 

Oct.  201 6C 

Number  of 
months 
difference11 

Seattle 

Building  101  Mental 
Health 

222,000,000 

192,424,000 

-13.3  June-2015 

September-2018 

39 

Seattle 

Correct  Seismic 
Deficiencies  in  Various 
Buildings 

51,800,000 

43,880,000 

-15.3  September-2015 

May-2016 

8 

St.  Louis 

Medical  Facility 
Improvement  and 
Cemetery  Expansion 

366,500,000 

366,500,000 

0.0  TBD 

August-2020 

NA 

Walla  Walla 

Multi  Specialty  Care 

71,400,000 

71,400,000 

0.0  January-2016 

March-2019 

38 

U.S.  Army  Corps  of  Engineers  (USACE)-managed  projects 

American 

Lakef 

New  Building  201, 
Building  81  Seismic 
Corrections  and 

Building  18  and  81AC 
Renovation 

NA 

161,700,000 

NA  NA 

TBD 

NA 

Canandaigua 

New  Construction  and 
Renovation 

370,100,000 

309,500,000 

-16.4  TBD 

TBD 

NA 

Denver 

New  Medical  Facility 

800,000,000 

1,675,000,000 

109.4  April-2015 

January-2018 

33 

Long  Beach 

Seismic  Corrections  to 
Mental  Health  and 
Community  Living 

Center 

258,400,000 

317,300,000 

22.8  TBD 

TBD 

NA 

Louisville 

New  Medical  Facility 

900,000,000 

925,000,000 

2.8  TBD 

TBD 

NA 

Palo  Alto 

Livermore  Realignment 

354,300,000 

415,600,000 

17.3  TBD 

TBD 

NA 

Sacramento 

Alameda  Outpatient 

Clinic 

208,600,000 

240,200,000 

15.1  TBD 

TBD 

NA 

San  Diego 

Spinal  Cord  Injury, 
Seismic  Corrections 

195,000,000 

227,100,000 

16.5  TBD 

TBD 

NA 

San 

Francisco 

Correct  Seismic 
Deficiencies,  Buildings 
1,6,  8,  and  12 

224,800,000 

346,700,000 

54.2  TBD 

TBD 

NA 

Tampa 

Polytrauma  Expansion 
and  Bed  Tower 

231,500,000 

231,500,000 

0.0  October-2011 

TBD 

NA 

West  Los 
Angeles 

Seismic  Corrections  to 
Various  buildings 

346,900,000 

370,800,000 

6.9  December-2013 

TBD 

NA 

Legend: 

TBD=to  be  determined 
NA=not  applicable 

Source:  GAO  Analysis  of  VA  data.  |  GAO-17-70 

aProjects  listed  as  NA  in  this  column  are  projects  whose  estimated  costs,  either  in  November  2012  or 
March  2016,  were  unknown. 

bProjects  with  TBD  listed  in  this  column  were  projects  whose  completions  dates  as  of  November  2012 
were  to-be-determined. 
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cThe  10  projects  whose  completion  dates  are  list  as  TBD  in  this  column  are  projects  whose 
management  VA  is  outsourcing  to  the  U.S  Army  Corps  of  Engineers  (USACE).  USACE  does  not  yet 
have  completion  dates  for  these  projects.  The  Denver  project  has  also  been  outsourced  to  USACE, 
but  USACE  was  able  to  provide  a  completion  date  for  this  project. 

^Projects  listed  as  NA  in  this  column  are  projects  whose  completion  dates,  either  currently  or  in 
November  2012,  were  to-be-determined  or  not  applicable. 

eThis  project  includes  phases  to  build  a  parking  garage  and  a  spinal  cord  injury  facility.  This  estimate 
is  for  the  parking  garage  phase  only.  The  spinal  cord  injury  phase  has  not  received  funding  so  there 
is  no  date  estimate  for  that  phase. 

'Officials  said  a  demolition  phase  was  added  to  this  project  after  Nov  2012,  which  extended  its 
completion  timeframe. 

9There  was  no  original  cost  estimate  published  for  this  project  at  the  time  construction  began. 

hOfficials  said  that  the  estimated  completion  date  does  not  include  Ambulatory  Care  Center  phase  of 
this  project  because  it  has  not  received  funding. 

'This  project’s  scope  was  expanded  in  fiscal  year  2015  to  include  additional  work.  The  estimated  cost 
and  completion  timeframe  in  November  2012  is  therefore  no  longer  applicable. 
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We  assessed  the  estimated  cost  to  complete  construction  of  the 
Department  of  Veterans  Affairs’  new  Denver  medical  center  using  the 
GAO  Cost  Estimating  and  Assessment  Guide’s  framework  of  the  four 
characteristics — comprehensive,  well-documented,  accurate,  and 
credible — associated  with  high-quality,  reliable  cost  estimates.1 
Specifically,  we  assessed  the  Denver  project’s  construction  cost  estimate 
against  the  best  practices  associated  with  these  four  characteristics.  The 
U.S.  Army  Corps  of  Engineers  developed  the  current  cost  estimate  to 
complete  construction  of  this  project.  Table  7  provides  greater  detail  of 
our  comparison  of  the  estimate  with  the  leading  practices  that  are  aligned 
with  the  four  cost  estimating  characteristics. 


^AO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs  (Supersedes  GAO-07-1 134SPJ,  GAO-09-3SP 
(Washington,  D.C.:  Mar.  2,  2009).  The  methodology  outlined  in  this  guide  is  a  compilation 
of  20  best  practices  that  federal  cost-estimating  organizations  and  industry  use  to  develop 
and  maintain  reliable  cost  estimates  throughout  the  life  of  a  government  acquisition 
program.  The  leading  practices  were  developed  in  conjunction  with  government  and 
industry  experts  in  the  cost-estimating  community  and  have  been  applied  in  past  work 
involving  federal  construction  projects.  By  default,  the  guide  also  serves  as  a  guiding 
principle  for  our  auditors  to  evaluate  the  economy,  efficiency,  and  effectiveness  of 
government  programs.  We  determined  that  most  of  the  leading  practices  were  applicable 
to  the  assessment  of  the  cost  estimate  for  completing  construction  of  the  Denver  project. 
However,  we  determined  that  the  best  practice  of  having  a  group  outside  the  agency 
conduct  an  independent  cost  estimate  to  not  be  applicable  to  the  construction  cost 
estimate  because  the  estimate  itself  served  as  an  independent  cost  estimate.  The 
purpose  of  the  estimating  effort  was  to  develop  an  independent  cost  estimate  that  would 
enable  the  VA/USACE  to  establish  a  firm  target  price  with  the  construction  contractor  for 
the  remaining  construction.  VA/USACE’s  cost  estimate  served  as  an  independent  cost 
estimate  for  comparison  with  the  construction  contractor’s  estimate,  so  the  criteria  for  an 
independent  cost  estimate  is  inapplicable. 
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Practices  for  Cost  Estimating 

Table  7:  GAO  Assessment  of  the  Cost  Estimate  to  Complete  Construction  of  the  Department  of  Veterans  Affairs  (VA)  Denver 
Medical  Center  Compared  to  the  Four  Characteristics  of  High-Quality,  Reliable  Cost  Estimates 

Characteristic 

Characteristic 

assessment 

Best  practice 

Individual  assessment  and  key  examples  of 
rationale 

Comprehensive 

Substantially 

meets 

The  cost  estimate  includes  all  costs 

Fully  meets:  All  applicable  costs  for  the 
construction  contract  appear  to  be  included  in  the 
cost  estimate. 

Completely  define  program,  reflect  current 
schedule,  and  be  technically  reasonable 

Substantially  meets:  The  cost  estimate  is  based 
on  an  assessment  of  all  remaining  construction 
work  on  the  project  to  be  completed.  However,  the 
technical  baseline  documentation  does  not 
discuss  costand  technical  risk 

The  cost  estimate’s  work  breakdown 
structure  is  product-oriented,  traceable  to 
the  statement  of  work/objective,  and  at  an 
appropriate  level  of  detail  to  ensure  that  cost 
elements  are  neither  omitted  nor  double- 
counted3 

Fully  meets:  The  work  breakdown  structure 
outlines  all  major  work  for  the  project. 

Document  all  cost-influencing  ground  rules 
and  assumptions 

Partially  meets:  The  estimate  details  all  ground 
rules  and  assumptions,  but  does  not  trace  risks  to 
specific  work  breakdown  structure  elements. 
Additionally,  while  inflation  was  incorporated  into 
the  estimate,  it  did  not  identify  the  source  of 
inflation  indexes. 

Well-documented 

Substantially 

meets 

Documents  capture  source  data,  their 
reliability,  and  how  they  were  normalized 

Partially  meets:  While  some  cost  estimating 
parameters  are  included  in  the  estimate’s 
supporting  documentation,  not  all  of  the  data  or 
data  sources  are  included. 

The  documentation  describes  in  sufficient 
detail  the  calculations  performed  and  the 
estimating  methodology  used  to  derive  each 
element’s  cost 

Substantially  meets:  Cost-estimating  methods 
used  include  bottom-up  and  parametric 
approaches,  but  the  estimate’s  documentation 
does  not  contain  historical  data  as  bases  of  the 
parametric  methodologies. 

The  documentation  describes  step  by  step 
how  the  estimate  was  developed  so  that  a 
cost  analyst  unfamiliar  with  the  program 
could  understand  what  was  done  and 
replicate  it 

Substantially  meets:  The  documentation  provides 
detailed  information  about  the  WBS  structure,  the 
cost-estimating  methodologies,  and  assumptions 
and  exclusions,  but  does  not  provide  step-by-step 
calculations  for  each  cost  element. 

Documents  discuss  technical  baseline 
description  and  that  the  data  in  the  baseline 
are  consistent  with  the  estimate 

Fully  meets:  The  estimate  was  based  on  an 
assessment  of  the  scope  of  work  remaining  at  the 
time  it  was  produced  and  represented  the 
technical  baseline. 

Provides  evidence  that  management 
reviewed  and  accepted  the  estimate 

Partially  meets:  USACE  officials  said  that  they 
provided  many  formal  briefings  to  management, 
but  they  did  not  provide  us  with  any  examples  of 
these  briefings. 
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Characteristic 

Characteristic 

assessment 

Best  practice 

Individual  assessment  and  key  examples  of 
rationale 

Accurate 

Substantially 

meets 

The  cost  estimate  results  are  unbiased,  not 
overly  conservative  or  optimistic  and  based 
on  an  assessment  of  most  likely  costs. 

Substantially  meets:  The  confidence  level  for  the 
base  estimate  is  calculated  as  less  than  10 
percent.  A  revised  version  of  the  cost  estimate, 
completed  in  August  2015,  did  not  have  an 
associated  updated  confidence  level. 

Adjusted  properly  for  inflation. 

Partially  meets:  We  could  not  verify  that  the 
estimate  was  properly  adjusted  for  inflation 
because  documentation  does  not  include 
calculations  involving  inflation  factors. 

The  estimate  contains  few,  if  any,  minor 
mistakes. 

Partially  meets:  USACE  officials  said  that  all  of  the 
cost  estimates  were  double  checked.  However, 
the  base  estimate  reported  in  the  risk  analysis 
could  not  be  found  in  other  reports  on  the 
estimate. 

The  cost  estimate  is  regularly  updated  to 
reflect  significant  changes  in  the  program  so 
that  it  is  always  reflecting  current  status. 

Substantially  meets:  USACE  officials  said  that 
when  changes  have  to  be  made  to  the  estimate, 
the  variations  are  explained  in  detail  in  various 
reports.  Several  changes  that  were  made  to  the 
estimate,  as  well  as  why  the  changes  were  made, 
were  documented. 

Variances  between  planned  and  actual  costs 
are  documented,  explained,  and  reviewed. 

Minimally  meets:  Officials  did  not  provide  a 
sufficient  explanation  of  how  they  track  variances 
between  actual  and  planned  costs. 

The  estimate  is  based  on  a  historical  record 
of  cost  estimating  and  actual  experiences 
from  other  comparable  programs. 

Fully  meets:  The  data  used  for  the  estimate  were 
from  primary  sources,  including  construction  plans 
and  specifications,  a  detailed  inspection  of  the 
remaining  work,  interviews  with  VA,  USACE,  and 
contractor  officials,  and  national  and  local 
vendors. 

Estimating  technique  for  each  cost  element 
was  used  appropriately. 

Fully  meets:  Officials  said  that  they  used  a 
bottom-up  estimating  technique.  For  this 
technique,  work  breakdown  structure  cost 
elements  were  defined  in  detail  by  a  work 
breakdown  structure  dictionary  using  current  and 
relevant  data  that  was  adequate  for  estimating 
element  costs. 

Credible 

Substantially 

meets 

Include  sensitivity  analysis  that  identifies  a 
range  of  possible  costs  based  on  varying 
major  assumptions,  parameters,  and  data 
inputs. 

Partially  meets:  While  a  risk  analysis  identifies  all 
key  cost  and  risk  drivers,  USACE  did  not  conduct 
a  formal  sensitivity  analysis. 

A  risk  and  uncertainty  analysis  was 
conducted  that  quantified  the  imperfectly 
understood  risks  and  identified  the  effects  of 
changing  key  cost  driver  assumptions  and 
factors. 

Substantially  meets:  USACE  performed  a  formal 
risk  and  uncertainty  analysis  and  developed  the 
most  likely  cost  of  each  of  the  risk  drivers  along 
with  a  minimum  to  maximum  range  for  variables. 
However,  an  updated  risk  and  uncertainty  analysis 
was  not  conducted  for  an  updated  version  of  the 
cost  estimate. 
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Characteristic  Characteristic  Best  practice 

assessment 

Individual  assessment  and  key  examples  of 
rationale 

Major  cost  elements  were  cross-checked  to 
see  whether  results  were  similar. 

Substantially  meets:  USACE  developed  a 
statistical  relationship  to  compare  costs  at  a  high 
level,  but  did  not  perform  cross-checks  of  major 
cost  elements. 

Independent  estimate  was  conducted  by  an 
outside  group  to  determine  whether  other 
estimating  methods  produced  similar  results. 

Not  applicable:  We  excluded  this  best  practice 
because  the  purpose  of  the  USACE’s  estimating 
effort  was  to  establish  a  firm  target  price  for  the 
contractor  to  complete  the  remaining  construction. 
USACE’s  estimate  served  as  an  independent  cost 
estimate  for  comparison  with  the  contractor’s 
estimate.  Therefore,  the  requirement  for  the 
independent  cost  estimate  is  not  applicable. 

Legend: 

Fully  met=VA/USACE  provided  complete  evidence  that  satisfies  the  entire  criterion 
Substantially  met=VA/USACE  provided  evidence  that  satisfies  a  large  portion  of  the  criterion 
Partially  met=VA/USACE  provided  evidence  that  satisfies  about  half  of  the  criterion 
Minimally  met=VA/USACE  provided  evidence  that  satisfies  a  small  portion  of  the  criterion 
Not  met=VA/USACE  provided  no  evidence  that  satisfies  any  of  the  criterion 

Source:  GAO  analysis  of  VA  and  USACE  data.  |  GAO-17-70 

Notes:  This  analysis  focused  only  on  the  cost  estimate  for  the  contractor  to  complete  construction  of 
the  facility.  The  estimate  does  not  include  all  costs  from  the  inception  of  the  program  through  design, 
development,  construction,  and  operation  and  maintenance  because  the  scope  was  defined  as 
developing  an  estimate  to  enable  the  government  to  establish  a  firm  target  price  for  the  remainder  of 
the  construction  contract  work. 

aA  work  breakdown  structure  is  supposed  to  define  in  detail  the  work  necessary  to  accomplish  a 
project’s  objectives. 
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’s  Schedule  Estimate  with  Best 

Practices 

We  assessed  the  schedule  estimate  to  complete  construction  of  the 
Department  of  Veterans  Affairs’  new  Denver  medical  center  using  the 

GAO  Schedule  Assessment  Guide’s  framework  of  the  four 
characteristics — comprehensive,  well-constructed,  credible,  and 
controlled — of  high-quality,  reliable  schedule  estimates.1  Specifically,  we 
assessed  the  Denver  project’s  schedule  estimate  against  the  best 
practices  associated  with  these  four  characteristics.  The  U.S.  Army  Corps 
of  Engineers  monitors  the  schedule  estimate  to  complete  construction  of 
this  project.  Table  8  provides  greater  detail  of  our  comparison  of  the 
estimate  with  the  leading  practices  that  are  aligned  with  the  four 
schedule-estimating  characteristics. 

Table  8:  GAO’s  Assessment  of  the  Schedule  Estimate  to  Complete  Construction  of  the  Department  of  Veterans  Affairs’  (VA) 
Denver  Medical  Center,  Compared  to  the  Four  Characteristics  of  High-Quality,  Reliable  Cost  Estimates 

Characteristic 

Characteristic 

assessment 

Best  practice 

Individual  assessment  and  key  examples  of  rationale 

Comprehensive 

Partially  Meets 

The  schedule  captures  all 
activities 

Partially  meets:  The  construction  schedule  appears  to 
include  all  work  necessary  to  complete  construction. 

However,  activities  in  the  schedule  are  not  consistently 
mapped  to  a  well-defined  work  breakdown  structure.3 

The  schedule  has  resources 
assigned  to  all  activities 

Not  meets:  There  are  no  resources  assigned  to  activities. 

The  schedule  establishes  the 
durations  of  all  activities 

Fully  meets:  The  information  provided  by  USACE  indicates 
their  confidence  that  activity  durations  were  developed 
carefully  and  have  been  vetted  and  monitored  by  USACE 
and  the  construction  contractor. 

Well-constructed 

Partially  Meets 

The  schedule  sequences  all 
activities 

Substantially  meets:  The  majority  of  activities  have 
appropriate  logic  and  the  use  of  constraints  is  reasonable. 
However,  the  schedule  includes  lag  on  296  activities  that  are 
not  justified  in  documentation.  Lags  denote  the  passage  of 
time  and  should  only  represent  a  real  need  to  delay  time 
between  activities. 

^AO,  GAO  Schedule  Assessment  Guide:  Best  Practices  for  Project  Schedules, 

GAO-1 6-89G  (Washington,  D.C.:  Dec.  22,  2015).  The  GAO  Schedule  Assessment  Guide 
presents  the  scheduling  concepts  introduced  in  the  Cost  Estimating  and  Assessment 
Guide  as  10  leading  practices  associated  with  developing  and  maintaining  a  reliable,  high- 
quality  schedule.  The  leading  practices  were  developed  in  conjunction  with  government 
and  industry  experts  in  the  schedule-estimating  community.  The  GAO  Schedule 
Assessment  Guide  serves  also  to  present  guiding  principles  for  our  auditors  in  evaluating 
the  economy,  efficiency,  and  effectiveness  of  government  programs. 
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Construction’s  Schedule  Estimate  with  Best 

Practices 

Characteristic 

Characteristic 

assessment 

Best  practice 

Individual  assessment  and  key  examples  of  rationale 

The  schedule  has  a  valid 
critical  pathb 

Substantially  meets:  The  longest  path,  which  USACE  uses 
in  place  of  the  critical  path,  is  valid  and  not  driven  by  lags  or 
constraints.  However,  the  activities  that  are  included  on  the 
longest  path  do  not  appear  to  include  major  works  such  as 
utilities,  systems,  electrical,  mechanical,  and  the  like. 

The  schedule  has  reasonable 
total  float0 

Minimally  meets:  The  schedule  appears  to  have  an 
excessive  amount  of  total  float.  For  example,  80  percent  of 
remaining  activities  are  able  to  slip  more  than  2  working 
months  before  affecting  the  key  milestone  date. 

Credible 

Partially  Meets 

The  schedule  can  be  traced 
horizontally  and  vertically 

Partially  meets:  Lower  levels  of  the  schedule  roll  up  to  higher 
work  breakdown  structure  levels.  However,  the  schedule 
logic  has  gaps  that  indicate  the  schedule  may  not  depict 
relationships  between  different  project  elements. 

A  schedule  risk  analysis  was 
conducted 

Partially  meets:  A  schedule  risk  analysis  was  conducted,  but 
key  details  of  the  analysis  are  not  available  in  the  provided 
documentation. 

Controlled 

Substantially 

meets 

The  schedule  is  updated 
using  actually  progress  and 
logic 

Substantially  meets:  The  schedule  is  updated  periodically 
and  delivered  to  the  project  management  team  monthly. 
Additionally,  our  analysis  found  no  date  anomalies  in  the 
schedule.  However,  there  is  no  accompanying  schedule 
narrative  that  documents  changes. 

A  baseline  schedule  is 
maintained 

Substantially  meets:  USACE  officials  provided  us  with  a 
baseline  schedule  that  we  confirmed  is  valid  and  produces 
baseline  dates  and  variances  when  compared  to  the  current 
construction  schedule.  While  there  is  no  accompanying 
schedule  basis  document,  officials  did  provide  some 
documentation  of  acronyms,  work  breakdown  structure 
elements,  and  schedule  ground  rules  and  assumptions. 

Legend: 

Fully  met=VA/USACE  provided  complete  evidence  that  satisfies  the  entire  criterion 
Substantially  met=VA/USACE  provided  evidence  that  satisfies  a  large  portion  of  the  criterion 
Partially  met=VA/USACE  provided  evidence  that  satisfies  about  half  of  the  criterion 
Minimally  met=VA/USACE  provided  evidence  that  satisfies  a  small  portion  of  the  criterion 
Not  met=VA/USACE  provided  no  evidence  that  satisfies  any  of  the  criterion 

Source:  GAO  analysis  of  VA  and  USACE  data.  |  GAO-17-70 

Note:  This  analysis  focused  only  on  the  schedule  estimate  for  the  contractor  to  complete  construction 
of  the  facility. 

aA  work  breakdown  structure  is  supposed  to  define  in  detail  the  work  necessary  to  accomplish  a 
project’s  objectives. 

bA  critical  path  is  the  sequence  of  activities  that  represents  the  longest  path  from  the  project’s  start 
and  finish  dates. 

cFloat  is  the  amount  of  time  by  which  an  activity  can  be  delayed  before  the  delay  affects  the  project’s 
estimated  finish  date. 
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Department  of  Veterans  Affairs 
Washington  DC  20420 


February  14,  2017 


Mr.  Dave  Wise 

Director,  Physical  Infrastructure 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Mr.  Wise: 

The  Department  of  Veterans  Affairs  (VA)  has  reviewed  the  Government 
Accountability  Office's  (GAO)  draft  report,  “VA  CONSTRUCTION:  Better  Processes 
for  Monitoring  Contract  Modifications,  Developing  Schedules,  and  Estimating 
Costs  Needed”  (GAO-17-70). 

The  enclosure  provides  our  general  and  technical  comments  and  sets  forth  the 
actions  to  be  taken  to  address  the  GAO  draft  report  recommendations. 

VA  appreciates  the  opportunity  to  comment  on  your  draft  report. 


Sincerely, 


Gina  S.  Farrisee 
Acting  Chief  of  Staff 


Enclosure 
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Department  of  Veterans  Affairs  (VA)  Comments  to 
Government  Accountability  Office  (GAO)  Draft  Report 

“VA  CONSTRUCTION:  Better  Processes  for  Monitoring  Contract  Modifications, 
Developing  Schedules,  and  Estimating  Costs  Needed” 

(GAO- 17-70) 

General  Comment: 

As  requested,  the  Department  of  Veterans  Affairs  (VA)  has  reviewed  the  subject 
Government  Accountability  Office  (GAO)  draft  report  and  concurs  with  its 
recommendations.  VA  appreciates  the  effort  undertaken  by  GAO  to  evaluate  the  VA 
performance  regarding  the  construction  of  VA  Facilities.  We  are  especially  appreciative 
of  GAO’s  acknowledgement  of  past  improvements  and  progress  at  improving  VA 
construction  management.  VA  is  consistently  engaged  in  efforts  to  improve 
construction  management,  and  has  acknowledged  the  evaluations  and 
recommendations  of  GAO  and  VA’s  Office  of  the  Inspector  General  (OIG).  VA’s  Office 
of  Acquisition,  Logistics,  and  Construction  (OALC),  Office  of  Construction  and  Facilities 
Management  (CFM),  has  put  in  place  sound  construction  management  processes 
based  on  best  practices  from  private  industry  and  other  Federal  agencies.  CFM  has 
also  engaged  in  lessons  learned,  including  those  from  the  Denver  project;  and 
recommendations  made  to  VA  from  various  stakeholders  including  GAO,  VA  OIG,  the 
United  States  Army  Corps  of  Engineers  (USACE),  and  construction  industry  partners. 
These  process  improvements  will  help  ensure  proper  execution  of  our  major 
construction  projects  and  future  success  throughout  the  VA  construction  program,  while 
allowing  VA  to  provide  increased  access  to  care  for  Veterans  and  their  families  around 
the  country.  This  draft  GAO  report  validates  that  improvements  occurred  and  additional 
improvements  are  underway.  VA  acknowledges  and  accepts  the  recommendations. 

Since  the  report  has  been  drafted,  several  updates  have  taken  place: 

a.  At  the  time  of  the  GAO  study,  VA  and  USACE  were  in  the  process  of  finalizing 
the  Enterprise  Program  Management  Plan.  This  plan  was  signed  December  2016. 

b.  As  stated  in  the  report,  the  most  current  Contract  Modification  Handbook  was 
issued  in  August  2013.  The  handbook  has  since  been  updated  as  a  result  of  a 
recommendation  by  VA’s  OIG.  The  updated  Contract  Modification  Handbook  is  now 
expected  to  be  finalized  and  distributed  by  the  end  of  the  second  quarter  of  fiscal 
year  2017. 

c.  The  report  noted  the  CFM  policy  stated  that  Regional  Acquisition  Directors  could 
approve  contract  actions  up  to  $2  million  without  further  administrative  review.  This 
limit  has  been  revised.  Now,  the  contracting  officer’s  authority  to  approve  contract 
actions  (to  include  change  orders)  has  increased  to  $5  million,  and  Regional 
Acquisition  Directors  can  approve  contract  actions  up  to  $15  million. 

d.  The  San  Francisco  Seismic  Correction  and  Research  Facility  project's  estimated 
cost  increased  by  54  percent  from  2012  to  2016.  While  CFM  has  achieved  35 
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Developing  Schedules,  and  Estimating  Costs  Needed” 

(GAO-17-70) 

percent  design  and  locked  the  requirement,  the  funding  plan  for  the  project  is  not 
defined.  The  cost  increases  are  a  result  of  escalation  cost  estimates,  increased 
space,  and  displacement  costs  during  renovation.  Specifically,  escalation  is  an 
element  of  cost  on  all  projects.  VA  adjusts  the  escalation  to  an  anticipated  mid-point 
of  construction  to  account  for  price  pressure  from  inflation  and  labor  market 
increases.  For  San  Francisco  ,  the  mid-point  of  construction  has  moved  from  2016 
to  2020.  VA  also  recently  identified  a  requirement  for  a  25,000  square  foot  increase 
in  the  research  space.  The  cost  increase  also  covers  additional  construction  impact 
requirements,  which  are  needed  to  accommodate  displaced  mission  functions, 
during  renovation.  The  cost  impacts  are  to  the  total  project  budget  and  not  the 
currently  appropriated  and  authorized  phase. 

e.  Referencing  page  14,  Table  2,  the  Palo  Alto  estimate  shown  was  not  updated  in 
2016  and  is  the  original  estimate.  VA  does  anticipate  this  estimate  to  increase  as  a 
result  of  escalation,  as  full  construction  funding  has  not  been  received.  VA  is 
evaluating  the  ability  to  offset  the  increases  through  scope  reductions  or  value 
engineering  efforts. 

f.  Referencing  page  14,  table  2,  construction  completion  timeframes  for  Palo  Alto 
and  St.  Louis  will  change  once  the  final  construction  funds  are  received.  The  June 
2019  date  for  Palo  Alto  and  the  August  2020  for  St.  Louis  are  dependent  upon 
construction  funding  being  received  in  2018. 

g.  VA  intended  to  procure  new  construction  management  software  by  March  of 
2017.  This  procurement  has  been  delayed.  VA  currently  plans  to  have  a  system  in 
place  to  improve  tracking  and  managing  modifications  by  the  March  2017  timeframe. 

h.  The  report  states  that  the  USACE  contractor  developed  construction  schedule 
only  partially  meets  characteristics  of  a  reliable  schedule.  We  agree  with  GAO  on 
the  need  to  have  a  reliable  schedule,  and  that  the  Denver  construction  schedule 
does  not  fully  meet  those  characteristics.  However,  VA  has  confidence  in  the 
current  Denver  schedule,  and  is  synchronizing  the  current  activation  schedule  with 
the  USACE  schedule.  VA  confidence  in  the  USACE  schedule  is  evidenced  by  the 
contractor  currently  meeting  or  beating  scheduled  building  turnover  dates  and  VA 
receiving  access  to  buildings  to  begin  activation..  USACE  has  responded  that  they 
normally  produce  schedules  that  meet  the  characteristics  of  a  reliable  schedule,  and 
Denver  is  an  anomaly  to  USACE  normal  process,  based  on  the  state  of  the 
construction  when  they  issued  the  new  construction  contract. 

VA  has  taken  steps  to  improve  the  construction  process  which  were  highlighted  in  the 
report.  GAO’s  acknowledgements  of  improvements  noted  in  the  report  are: 
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a.  VA  defined  roles  and  responsibilities  of  the  project  team  members  to  reduce 
confusion  in  the  execution  phase.  GAO  found  evidence  of  improved 
communications  and  relationships  between  VA  and  the  contractor. 

b.  VA's  integration  of  full  time  Medical  Planners  on  the  project  team  is  occurring  to 
assist  in  coordination  of  design  with  medical  equipment  and  reduce  time  and  cost . 
While  the  evidence  may  be  slow  to  show,  the  reality  is  we  are  able  to  take  time  and 
cost-savings  action  if  a  delay  is  detected. 

c.  VA’s  efforts  to  streamline  the  change  order  approvals  have  reduced  time  for 
processing,  and  we  are  seeing  the  results  in  comments  from  our  contractors.  Our 
ability  to  keep  the  contract  current  (modifications  approved  in  a  timely  manner) 
demonstrates  a  level  of  commitment  to  our  contractors  that  shows  in  improved 
relationships. 

d.  VA  issued  guidance  on  Framework  Principles  for  the  Delivery  of  Major 
Construction  Projects.  This  guidance  establishes  clean  lines  of  authority  for 
execution  of  design  and  construction  and  established  the  Executive  Director,  CFM 
as  the  responsible  authority  for  changes  to  the  design  of  projects.  This  provides 
clear  governance  of  the  construction  process. 

e.  VA  issued  guidance  on  Foundation  of  Project  Design  Principles.  This  guidance 
lays  the  framework  to  assure  construction  projects  are  functional  and  that  aesthetics 
do  not  govern  the  design. 

f.  VA  issued  guidance  on  changes  requested  by  VA  medical  staff  to  improve 
coordination  and  control.  This  guidance  establishes  the  Executive  Director,  CFM,  as 
the  responsible  authority  for  incorporation  of  medical  center  requested  changes. 

These  and  many  other  improvements  are  targeted  to  improve  design  and  construction 
management,  which  promotes  cost  control  and  limits  schedule  growth.  Design  and 
construction  is  a  lengthy  process,  and  the  results  of  these  efforts  may  not  show  their  full 
benefits  for  several  more  years,  particularly  as  improvements  in  planning  will  not  show 
until  well  into  construction,  often  4  to  10  years  later  VA  recognizes  that  improvement  of 
the  design  and  construction  process  is  iterative  and  is  committed  to  continuous  process 
improvements. 

VA  acknowledges  the  recommendations  made  by  GAO  and  is  working  to  implement  the 
recommendations.  Our  plan  to  implement  the  recommendations  follows. 
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VA  is  strongly  committed  to  developing  a  reliable  activation  cost  estimate  that  conforms 
to  best  practices  to  allow  Congress  and  VA  for  informed  decision-making.  The  content 
in  this  draft  report  applies  to  high  risk  area  2  (inadequate  oversight  and  accountability). 
VA  will  use  GAO’s  findings  to  continue  to  make  improvements  and  fulfill  VA’s  mission  of 
honoring  America’s  Veterans  by  providing  exceptional  health  care  that  improves  their 
health  and  well-being. 
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GAO  Recommendation:  To  improve  VA’s  management  of  medical  facility 
construction  projects  and  its  accountability  and  to  allow  for  more  informed 
decision  making  by  Congress  and  VA,  GAO  recommends  that  the  Secretary  of 
Veterans  Affairs  take  the  following  three  actions: 

Recommendation  1:  Establish  a  mechanism  to  monitor  the  extent  that  major 
facilities  projects  are  following  guidelines  on  change  order  timeframes  and 
design  changes. 

VA  Comment:  Concur.  The  Department  of  Veterans  Affairs  (VA)  has  developed  a 
plan  to  establish  procedures  and  processes  for  monitoring  Construction  Facilities 
Management  compliance  with  the  timeframes  established  in  the  updated  Contract 
Modification  Handbook.  The  process  will  standardize  the  reporting  and  will  use  existing 
reporting  structures  such  as  VA's  Project  Review  Board  to  assure  there  is  management 
oversight  of  the  compliance  to  the  time  standards  for  processing.  VA  will  have  the 
process  in  place  by  the  end  of  March  2017. 

Recommendation  2:  Develop  an  activation  cost  estimate  for  the  Denver  project 
that  is  reliable  and  conforms  with  best  practices  as  described  in  the  GAO  Cost 
Estimating  and  Assessment  Guide. 

VA  Comment:  Concur.  This  recommendation  is  related  to  high  risk  area  2  (inadequate 
oversight  and  accountability).  Developing  a  reliable  activation  cost  estimate  that 
confirms  with  best  practices  will  improve  VA’s  management  of  medical  facility 
construction  projects  and  will  allow  for  more  informed  decision  making  by  Congress  and 
VA. 

VA  concurs  with  having  a  reliable  cost  estimate  for  construction  projects.  To  provide 
rigor  and  improve  accuracy  of  activation  cost  estimates,  the  Veterans  Health 
Administration  (VHA)  developed  the  Activation  Cost  Budget  Model  (ACBM)  to  estimate 
Major  Construction  and  Major  Lease  activation  project  requirement  costs;  i.e.,  recurring 
and  non-recurring  activation  costs. 

The  development  of  activation  project  estimates  is  a  coordinated  effort  between  VA 
Medical  Centers  (VAMCs),  and  VHA’s  Office  of  Capital  Asset  Management  Engineering 
and  Support  (OCAMES).  Information  is  gathered  to  develop  preliminary  recurring  and 
non-recurring  activation  cost  estimates  for  new  space.  As  the  project  progresses 
through  the  acquisition  process,  estimates  continue  to  be  refined  through  the  use  of 
“progress  elaboration.”  Due  to  the  number  of  years  between  when  a  preliminary 
budgetary  estimate  is  prepared  and  when  a  project  nears  its  actual  activation  date, 
some  variation  is  expected  as  project  requirement  are  refined. 
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In  fiscal  year  (FY)  2012,  VHA  started  to  utilize  the  ACBM  to  provide  project  specific  cost 
estimates  to  bring  the  new  space  into  full  operational  status.  Since  the  development  of 
the  ACBM,  OCAMES  has  facilitated  field  data  calls  to  gather  all  needed  project  data  to 
run  the  model;  for  example,  Budget  Year,  Project  Type,  Space  Gap  Data,  Utilization 
Gap  Data,  Net  New  Full  Time  Equivalents  resulting  from  project,  Project  Duration, 
Expected  year  of  groundbreaking,  Square  Footage  included  in  project,  Additional 
Workload  Resulting  from  Project  -  Amount  of  workload  that  is  expected  to  be  transferred 
to  VA  from  a  fee  care  provider  as  a  result  of  the  project,  etc.  Additionally,  the  ACBM 
contains  assumptions  regarding  the  timing  of  costs  based  on  historical  VA  experience, 
and  calculates  recurring  costs  based  on  the  total  incremental  workload  and  non¬ 
recurring  linear  cost  per  square  foot,  by  department.  Activation  costs  are  phased 
across  the  multiple  years  of  each  project,  and  phasing  varies  depending  on  the  number 
of  years  of  construction. 

To  further  improve  the  accuracy  of  activation  costs,  in  FY  2015,  OCAMES-Activations 
developed  the  Activating  Funding  Process  (see  Figure  1).  This  process  utilizes  top- 
down  and  bottom-up  cost  development  methods.  Integral  to  this  process  is  the  analysis 
of  the  two  cost  estimates,  analyzing  the  variation,  validation  of  the  requirement  and 
expected  year  of  execution.  During  this  assessment,  the  ACBM  cost  estimates  are 
compared  to  the  project  requirements  developed  by  the  local  project  team;  which  results 
in  a  three  year  outlook  and  total  estimated  activation  cost.  As  needed,  adjustments  are 
made  to  the  original  estimates  due  to  variations  as  a  result  of  unknown  design  impacts 
on  operations,  variation  in  the  cost  of  furniture,  fixtures  and  equipment,  scope 
refinement,  specification  omissions  or  changes,  operation  cost  increases,  etc. 

Since  FY  2015,  Denver’s  $341  million  estimated  activation  cost  has  been  reassessed 
as  part  of  this  yearly  process,  and  determined  reasonable  to  bring  this  facility  into 
operational  status.  The  assessment  included  a  review  of  the  requirements,  justification 
and  comparison  to  multiple  ACBM  runs. 

As  part  of  FY  2015  improvements,  OCAMES  -  Activations  took  a  portfolio  management 
approach  to  support  field  operations,  by  offering  VHA  wide  services  and  tools.  This 
support  includes  access  to  activation  subject  matter  experts  to  assist  in  cost  estimating, 
activation  specific  contract  support,  assessment  tools,  and  contract  vehicles  to  assist 
VAMC  leadership  with  planning  and  managing  Major  Construction  and  Major  Lease 
Activation  Projects.  In  addition,  Clinical  and  Administrative  Activation  Checklists  have 
and  continue  to  be  developed  to  assist  the  field  operations  in  determining  requirements, 
and  ensuring  key  operational  components  are  addressed.  Activation  projects  teams 
have  access  to  the  Attainia  Database  System  to  plan  furniture,  fixtures  and  equipment, 
and  improve  the  accuracy  of  the  project  cost  estimates. 
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In  FY  2016,  an  analysis  was  conducted  on  the  ACBM  tool  and  activation  funding  process, 
to  improve  cost  forecasting  and  management  of  the  activations  process.  The  ACBM  has 
been  modified  to  better  support  short  and  long-term  funding  estimates  and  to  assist  VFIA 
with  the  portfolio  management  for  all  Major  Construction  and  Major  Lease  activations 
across  the  system.  New  features  include  project  tracking,  requirements  validation,  data 
versioning,  monthly  updating  capabilities,  and  robust  reporting  capabilities.  The  new  tool 
is  currently  scheduled  to  be  deployed  by  the  end  of  February  2017.  The  status  is 
complete. 
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Department  of  Veterans  Affairs  (VA)  Comments  to 
Government  Accountability  Office  (GAO)  Draft  Report 

“VA  CONSTRUCTION:  Better  Processes  for  Monitoring  Contract  Modifications, 
Developing  Schedules,  and  Estimating  Costs  Needed” 

(GAO-17-70) 

Recommendation  3:  Clarify  CFM  policies  to  require  that: 

•  all  projects  have  an  integrated  master  schedule  to  ensure  that  the 
integrated  master  schedules  include  and  link  all  construction  and 
activation  activities;  and 

•  the  policies  on  integrated  master  schedule  for  CFM  and  U.S.  Army 
Corps  of  Engineers  managed  projects  are  consistent. 

VA  Comment:  Concur.  VA  will  clarify  the  Policy  Memorandum  and  Standard 
Operation  Procedure  (SOP)  to  reinforce  the  policy  that  all  projects  develop  and  maintain 
an  Integrated  Master  Schedule  (IMS).  VA  is  developing  reports  for  the  organization  that 
demonstrate  the  IMS  is  linked  to  construction  and  activation,  and  also  provides 
management  with  meaningful  data  on  which  to  act  or  guide  the  project  team.  These 
reports  will  be  documented  in  SOPs  and  reviewed  by  leadership  to  ensure  the  project 
team  is  working  to  maintain  schedule. 

Denver  project  is  the  only  project  that  is  integrating  construction  and  activation 
schedules  into  the  IMS.  As  we  identify  the  lessons  learned,  IMS  SOPs  will  be 
developed.  The  SOPs  will  address  the  process  for  VA/United  States  Army  Corps  of 
Engineers  projects  as  well  as  improved  integration  of  the  activation  activities.  VA 
anticipates  completion  of  this  recommendation  by  July  2017. 
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Department  of  Veterans  Affairs  (VA)  Comments  to 
Government  Accountability  Office  (GAO)  Draft  Report 

“VA  CONSTRUCTION:  Better  Processes  for  Monitoring  Contract  Modifications, 
Developing  Schedules,  and  Estimating  Costs  Needed” 

(GAO-17-70) 

Technical  Comments: 

Page  9,  under  Figure  2,  first  bullet:  Bullet  reads: 

“VA  and  USACE  are  in  the  process  of  finalizing  an  Enterprise  Program 
Management  Plan  to  provide  a  framework  and  consistent  approach  for  USACE 
to  support  VA’s  design  and  construction  program  in  the  future.  The  plan  will 
formalize  USACE’s  role  and  align  with  the  underlying  goal  of  the  agencies' 
interagency  agreement  to  deliver  medical  facility  projects  with  cost  and  on 
schedule. " 

VA  Recommended  Edit:  Revise  bullet  to  read: 

“On  December  21,  VA  and  USACE  executed  an  Enterprise  Program 
Management  Plan  to  provide  a  framework  and  consistent  approach  for  USACE 
to  support  VA’s  design  and  construction  program  in  the  future.  The  plan 
formalizes  USACE’s  role  and  align  with  the  underlying  goal  of  the  agencies’ 
interagency  agreement  to  deliver  medical  facility  projects  within  cost  and  on 
schedule.  ” 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and  investigative 
arm  of  Congress,  exists  to  support  Congress  in  meeting  its  constitutional 
responsibilities  and  to  help  improve  the  performance  and  accountability  of  the 
federal  government  for  the  American  people.  GAO  examines  the  use  of  public 
funds;  evaluates  federal  programs  and  policies;  and  provides  analyses, 
recommendations,  and  other  assistance  to  help  Congress  make  informed 
oversight,  policy,  and  funding  decisions.  GAO’s  commitment  to  good  government 
is  reflected  in  its  core  values  of  accountability,  integrity,  and  reliability. 
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GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost  is 
through  GAO’s  website  (http://www.gao.gov).  Each  weekday  afternoon,  GAO 
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Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card,  MasterCard, 

Visa,  check,  or  money  order.  Call  for  additional  information. 
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E-mail:  fraudnet@gao.gov 
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Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512-4400, 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125, 
Washington,  DC  20548 
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